search

input-output-hk / iodb

Multiversioned key-value database, especially useful for blockchain
Creative Commons Zero v1.0 Universal
95 stars 13 forks source link

Expose incremental journal replay #25

Open publicocean0 opened 6 years ago

publicocean0 commented 6 years ago

I have a counter to update every add and remove . This counter is in memory for evoiding overhead in writes . A periodic thread save the values in db . The problem is if there is crash before last update is done . For solving it i might set the limit for compactation . Pseudo code Update counter and snapshot sequence values in a kv cell. Update the tail limit for compactation(go forward )

If there is a crash the last part of log file is not re compacted . On bots trap It is possible to scan db from last sequence assoacied to counter(surely there are few rows )i can reads add and remove commands and I can restore very Fastly the correct status of counter without scanning entire db

jankotek commented 6 years ago

I think I understand. You want to be able to replay changes made in last N versions. It is already possible to keep last N versions in journal (keepVersions parameter).

publicocean0 commented 6 years ago

hi it is interesting :)