tikv / raft-engine

A persistent storage engine for Multi-Raft log
Apache License 2.0
564 stars 88 forks source link

Atomic group causes data loss #315

Closed tabokie closed 1 year ago

tabokie commented 1 year ago

Using dump tool, we locate the missing entry is in file X. File X is the oldest rewrite log available. Use dump tool to analyze file X, it only contains AtomicGroup::Middle but no AtomicGroup::Begin.

What happened is, memtable is not aware of the atomic group that each entry/kv belongs to. After user deletes some of the entries inside an atomic group, memtable will permit the purge thread to delete the physical files that contain these entries. As a result, the Begin marker of this atomic group is deleted. Next restart this group will be broken and all entries including those not deleted by user will disappear.