lostatc / acid-store

[UNMAINTAINED] A transactional and deduplicating virtual file system
Apache License 2.0
95 stars 10 forks source link

How does this project relate to e.g. LSM implementations? #10

Open Qqwy opened 2 years ago

Qqwy commented 2 years ago

First of all, this is an amazing project. I am very impressed with how many different back-ends are supported right now, and how readable most of the source code is! :+1:

I have one question about how acid-store works, with relation to other database/datastore-like systems that for instance use a 'log structured merge tree' implementation where incoming transactions are added to a write-ahead log (for consistency/durability) as well as an in-memory dictionary (for speed of lookups using recent data), and periodically merge (some or all of) this data into the blocks stored on the permanent back-end.

Is this similar to what acid-store does as well? Or does acid-store always immediately (when calling .commit()) perform an update to the particular blocks in the back-end that require a change?

Put differently: Does acid-state perform any kind of amortization to make the average insert/update faster, or not?

whymidnight commented 2 years ago

Or does acid-store always immediately (when calling .commit()) perform an update to the particular blocks in the back-end that require a change

it seems as if this library computes a delta state change from the block used when constructing an operation and "prior-to-commitment" state. so there is nothing to amortize (since its derived as code)? i believe on .commit(), it is an immediate write to the store.

i plan on experimenting with this lib for a pricing engine and hopefully report back how it fared in production haha.

on a random note, i think the library implements something eerily similar to solana without a proof of work consensus layer?