tonbo-io / tonbo

A portable embedded database using Arrow.
https://tonbo.io
Apache License 2.0
606 stars 42 forks source link

Feat: Cleaner implements persistence #123

Closed KKould closed 2 weeks ago

KKould commented 2 weeks ago

Feature Request

https://github.com/tonbo-io/tonbo/blob/main/src/version/cleaner.rs Cleaner will record the newly added and deleted SSTables after each compaction, and delete the SSTables that are no longer needed after the relevant transaction is destroyed.

However, when the db program exits abnormally, Cleaner may cause the SSTable that needs to be deleted to be lost (because there is no persistence), so it may no longer process the SSTable that should be deleted after restarting.

I think Cleaner can persist this part of information to ensure that unnecessary SSTables can still be cleared after an abnormal restart.