DiceDB / dice

DiceDB is hyper-optimized for building and scaling truly real-time applications on modern hardware. It is a drop-in replacement for Redis with support for SQL-based reactivity.
https://dicedb.io/
Other
3.32k stars 420 forks source link

Ensure thread-safe access to store and keypool #173

Closed JyotinderSingh closed 1 month ago

JyotinderSingh commented 1 month ago

With the addition of the WatchKeys goroutine (to enable real-time reactive query subscriptions), concurrent reads can happen on the store and keypool maps.

This change adds RW locks to guard access to these data structures to prevent race conditions.

Benchmarks

These benchmarks test existing workflows which are mostly single-threaded, we may see some unavoidable slowdowns once we benchmark more sophisticated scenarios (involving multiple clients, some of which are subscribing to existing queries.

Overall, the performance is quite close to the thread-unsafe implementation, and the time taken seems to have increased within reason.

Thread-Safe Implementation

Benchmark Iterations Time (ns/op) Memory (B/op) Allocations (allocs/op)
BenchmarkInsert20-10 2,103,750 557.2 880 30
BenchmarkInsert200-10 277,537 4,245 5,016 215
BenchmarkInsert2000-10 26,451 45,406 68,808 2,048
BenchmarkInsertLL20-10 2,076,896 578.7 1,176 23
BenchmarkInsertLL200-10 220,178 5,270 11,448 203
BenchmarkInsertLL2000-10 18,030 66,256 126,393 3,747
BenchmarkInsertBasic20-10 6,297,427 189.8 536 7
BenchmarkInsertBasic200-10 1,241,326 982.1 4,120 10
BenchmarkInsertBasic2000-10 131,550 9,225 60,056 15
BenchmarkInsertRemove-10 2,791,825 429.7 324 22
BenchmarkSize-10 3,456,609 345.0 324 22

Thread-Unsafe Implementation (original)

Benchmark Iterations Time (ns/op) Memory (B/op) Allocations (allocs/op)
BenchmarkInsert20-10 2,093,487 559.7 880 30
BenchmarkInsert200-10 279,596 4,152 5,016 215
BenchmarkInsert2000-10 26,296 45,113 68,808 2,048
BenchmarkInsertLL20-10 2,081,374 595.3 1,176 23
BenchmarkInsertLL200-10 223,196 5,247 11,448 203
BenchmarkInsertLL2000-10 18,140 66,432 126,393 3,747
BenchmarkInsertBasic20-10 6,312,344 187.5 536 7
BenchmarkInsertBasic200-10 1,235,007 970.5 4,120 10
BenchmarkInsertBasic2000-10 125,288 9,224 60,056 15
BenchmarkInsertRemove-10 2,746,761 433.4 324 22
BenchmarkSize-10 3,437,666 347.7 324 22