arrdem / shelving

A toolkit for building data stores.
Eclipse Public License 1.0
38 stars 2 forks source link

Transaction/Pipeline API #19

Open arrdem opened 6 years ago

arrdem commented 6 years ago

Right now, all the Shelving stores sorta assume that they've got some in-memory state that's backed by a file which we may want to flush to ... eventually. Hence the flush operation. For networked stores, it'd be a good idea to have a transactions API which lets users batch writes for efficiency. Especially since content hashing means that we aren't dependent on a central system to dictate identifiers.

This also opens the gate to extending Shelving's model of tuples to include as Datomic does the transaction in which a tuple was created. Not something I have a use case for, but a possible neat addition.

This would suggest exposing some sort of write batching endpoint for implementations.

Being able to choose abort vs. clobber in the case of conflicts on records would be interesting. Most of Shelving actually is just the set monoid under addition so transaction conflicts should be rare.