search

facebook / rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.
http://rocksdb.org
GNU General Public License v2.0
28.4k stars 6.29k forks source link

Alternative layouts for SST files #9397

Open ghost opened 2 years ago

ghost commented 2 years ago

@siying shared a post about HTAP. It reminded me of the Ressi data layout in Spanner, which is based on PAX.

After moving from local storage to cloud storage, all SST files are stored in the cloud filesystem. Similar to the design of remote compaction contributed by @jay-zhuang, maybe the distributed processing frameworks like Spark can be used to convert the existing SST file formats to some alternative format like PAX for experiments.

lni commented 2 years ago

@cheng-chang

sounds like a very interesting topic, any chance to share the HTAP post from @siying ? Thanks!