This style in database storage engines is known as Log Structured data files. In this section will discuss some examples of data structures commonly used in storage engines, but let's first take a quick look at the simplest database you can build, you can actually build this using a file and 2 bash functions as follows: #!/bin/bash db_set () It also supports a variety of index types which have different speeds and limitations, we discussed this here as well. MySQL has very simple storage engines like a csv engine and very complex ones like InnoDB. Also some storage engines are simple and fast but only work with a small dataset while other storage engines are way more complex but can handle loads of data efficiently.Ī good example for how different storage engines can be within the same database is MySQL which has a variety of storage engines we discuss here before. In particular, there is a big difference between storage engines that are optimized for transactional workloads and those that are optimized for analytics. While this might look like getting into a low level territory of dealing with database internals, application developers who don't work with database internals directly still need to understand the main differences between data structures used in the storage engines of different databases to be able to not just select the suitable database for their application business usecase, but also the suitable storage engine for their database. Why Should I as an Application Developer care about Storage Engines? In later articles we will discuss more topics related to storage and retrieval, mainly, transactions processing and column-oriented storage.

This chapter is pretty long so we will discuss it over a series of articles, this one will discuss the data structures commonly used for storage such as hash indexes, SSTables, LSM-trees, B-trees, etc. Now we move on to chapter 3 which discusses storage and retrieval. In the previous two articles, we discussed chapter 2 of Designing Data Intensive Applications by Martin Kleppmann which discussed data models and query languages, here and here.

Kleppmann data intensive