search

apache / kvrocks

Apache Kvrocks is a distributed key value NoSQL database that uses RocksDB as storage engine and is compatible with Redis protocol.
https://kvrocks.apache.org/
Apache License 2.0
3.6k stars 470 forks source link

Optimize transactions for Multi/Exec commands #2554

Open jonathanc-n opened 2 months ago

jonathanc-n commented 2 months ago

Search before asking

Motivation

Issue referencing #2331 Want to optimize concurrency when executing transactions with multiple commands.

Solution

Possibly implement OCC or some other concurrency while not introducing too much complexity to the codebase.

Are you willing to submit a PR?

jonathanc-n commented 2 months ago

@PokIsemaine You can mark this in your issue you posted

PokIsemaine commented 2 months ago

Thanks for the reminder, I'm sorry for the slow progress due to my busy work lately.

My initial idea was to provide a snapshot-like mechanism based on the recently added engine::Context and apply it to Multi/Exec. A single engine::Context will manage multiple commands and replace exclusive locks compared to normal commands.

image

Now to consider:

  1. engine::Context is still experimental and unstable, do we want to support the new Multi/Exec pattern based on it now?
  2. An implementation based on engine::Context may improve concurrency, but at the cost of not achieving a serializable level of isolation, which I'm not sure fits into Multi/Exec's semantics

The above is my current thoughts, if you have other better suggestions, please also welcome to discuss.