Epic: Storage as an LSM tree

[hlinnaka]

Status

If you squint a little, our layered repository format already forms the bottom layer of an LSM tree. And that bottom layer is partitioned by SegmentTag. To turn that into a proper LSM tree, we need to make that partitioning dynamic, so that each layer can hold a range of SegmentTags, and add the upper levels.

Tasks on top of https://github.com/zenithdb/zenith/pull/1350:

• [x] #1355
• [x] #1356
• [x] #1357
• [x] #1358
• [ ] #1359
• [x] #1493

Other Tasks

• [ ] #1267
• [ ] #931
• [ ] Partition the key space statically. System catalogs are mostly bundled together into one partition, and other rels can be in separate partitions
• [ ] Partition the key space dynamically, based on relation sizes. Target is 1 GB of key space in each partition
• [ ] Add Level 1 of the LSM tree, on top of the bottom layer we have now. Implement compaction from Level 1 to bottom level
• [ ] Add support for more levels

[stepashka]

this could influence the implementation of #999

[hlinnaka]

Related presentation from December: https://docs.google.com/presentation/d/1sWBECwju7JYkk1U-xQpZ0ttlwwLX6PWCrig4SUfNR5s/edit?usp=sharing

@kelvich

[stepashka]

a related PR for detecting whether stored data is compatible with the running binary: https://github.com/zenithdb/zenith/pull/1379