Shannon-Data / ShannonBase

A MySQL HTAP Database, Open Source version of MySQL Heatwave, Powered by AI.
https://www.shannonbase.org
Other
14 stars 6 forks source link

feat(shannon): Multi-layer Storage Hierarchy #224

Open ShannonBase opened 1 month ago

ShannonBase commented 1 month ago

Summary

Description for this feature.

Using multi-layer storage hierarchy strategy to keep the balance between data volumn and performance.

layer 0                     layer 1                 layer 2
------------            ---------------        ---------------
  memory     <--->      high speed          <-->    disk
                               disk
------------           ----------------       ---------------

The warm data can be offloaded to layer 1 disk or loaded from layer 1 disk.

for example,we can create a tablespace for warm data, which located on layer1 disk. Due to layer1 is high speed disk(NVME, etc.), we will achieve a bettern performance.

CREATE TABLESPACE ts1 ADD DATAFILE ts1.ibd '/path/to/high-speed-disk/../ts1.ibd';