search

lucaong / cubdb

Elixir embedded key/value database
Apache License 2.0
556 stars 23 forks source link

Atomic gets that depend on each other #27

Closed refi64 closed 2 years ago

refi64 commented 3 years ago

TLDR what I want to do is something like this, but atomic:

k2 = CubDB.get(db, {:index, k1})
v = CubDB.get(db, {:stuff, k2})

In other words, I'm taking the value of one key and using it as part of another key, which doesn't seem to be possible with current APIs. Now for my use case the atomicity isn't essential, so I could just use the double gets. However, some support for this would be nice to have regardless...

lucaong commented 3 years ago

Hi @refi64 , you are right in that, while it is possible to get multiple "independent" entries atomically, there is currently no API to atomically get two entries, with the second key depending on the first. I will think about this use case, it would be great to enable it, as it could be especially useful for secondary indexes (as you seem to be doing in your case).

lucaong commented 2 years ago

Hi @refi64 , I know this comes quite a long time after this issue was open, but version v2.0.0 (currently still an unreleased work in progress) should meet your original need by introducing snapshots.

Snapshots are immutable read-only representations of the database at a specific point in time. They are basically zero-cost: nothing needs to be copied or written, apart from some small in memory internal bookkeeping.

Here's how you could solve your case once v2.0.0 will be released:

{k2, v} = CubDB.with_snapshot(db, fn snap ->
  k2 = CubDB.Snapshot.get(snap, {:index, k1})
  v = CubDB.Snapshot.get(snap, {:stuff, k2})

  {k2, v}
end)

Now, both reads are performed on the same immutable snapshot, isolated from any write that might have happened between them. Snapshots do not block concurrent read or write operation.

In cases when one also needs to write to the database, v2.0.0 will also introduce atomic transactions with arbitrary operations:

CubDB.transaction(db, fn tx ->
  # Perform dependent reads
  k2 = CubDB.Tx.get(tx, {:index, k1})
  v = CubDB.Tx.get(tx, {:stuff, k2})

  # Update the value
  tx = CubDB.Tx.put(tx, {:stuff, k2}, v + 10)

  # Commit the transaction
  {:commit, tx, :ok}
end)

Transactions do not block concurrent readers, but will block writers, so they should be used only when snapshots are not enough.

lucaong commented 2 years ago

This is now in v2.0.0-rc.1, and will soon be released as part of v2.0.0