search

linkedin / PalDB

An embeddable write-once key-value store written in Java
Apache License 2.0
939 stars 178 forks source link

make paldb reader thread safe #33

Closed xiaoxxcool closed 6 years ago

xiaoxxcool commented 7 years ago

I made some efforts to make paldb reader works in multi-thread environment, the perfermance decreased about 20% in single thread read, the perfermance test report is

Before change:
FILE LENGTH;    KEYS;       RPS
1831        100     3291755
16828       1000        2999658
166741      10000       2778105
1664749     100000      2019042
16643345    1000000     1587133
166431370   10000000    1424029

After change:
FILE LENGTH;    KEYS;       RPS
1831        100     2619438
16828       1000        2394381
166741      10000       2167783
1664749     100000      1577364
16643345    1000000     1388372
166431370   10000000    1192194

the benefit is you can share the StoreReader instance without lock.

bowenli86 commented 7 years ago

Just curious. Isn't StoreReader read only? Why does it need to be thread safe?

xiaoxxcool commented 7 years ago

my server contains some data update periodically, each request is processed in different threads, when the data is taken from the shared StoreReader instance, lock is required, or it will get the wrong data. what is a good practice to use paldb in such circumstance?