search

tonbo-io / fusio

Fusio provides file operations on multiple storages across various async runtimes.
Apache License 2.0
205 stars 8 forks source link

Feat: support copy file #99

Closed KKould closed 1 week ago

KKould commented 2 weeks ago

If the snapshot is on the same filesystem as the original database, the SST files will be hard-linked, otherwise SST files will be copied

https://rocksdb.org/blog/2015/11/10/use-checkpoints-for-efficient-snapshots.html

CheckPoint will "copy" the WAL corresponding to MemTable and the related SSTable in Version. However, tonbo supports multiple FileSystems based on fusio. I think fusio can copy file to solve this problem.

ethe commented 2 weeks ago

fusio::Fs should provide a unified "copy file" semantic, "try hard-linked first otherwise copy" may be a good choice. To those platorms that do not support hard-link, falling back to copy would never be a wrong behavior.

KKould commented 2 weeks ago

the semantics of hard_link and copy_file are not consistent. For example, when the files of hard_link are modified, the contents will change together, but copy_file will not.

ethe commented 2 weeks ago

It is not a good formal name, maybe we can choose another one, but the semantic would be good to me.