Congyuwang / RocksDict

Python fast on-disk dictionary / RocksDB & SpeeDB Python binding
https://congyuwang.github.io/RocksDict/rocksdict.html
MIT License
173 stars 8 forks source link

Support for entity (wide colums) batch put/get #131

Closed afbarbaro closed 2 months ago

afbarbaro commented 2 months ago

Hi, thank you for this project! I have found it super helpful and really easy to follow the documentation. ⭐⭐⭐⭐⭐ rating.

I'm currently using WriteBatch to write many keys at once, which is necessary for having good performance on my bulk loads. I'm also using the multi-get (get with a list of keys). I now want to try using entities (aka "wide columns") but I don't see the equivalent batch write and multi-get operations. They exist in RocksDB, so any chance these could be exposed in RocksDict soon?

Again, thank you for this project!

Congyuwang commented 2 months ago

Yeah right. Just like the current wide columns support, I'll need to first add an upstream PR to rocksdb to add C-API to support WriteBatch with entities.

I will probably work on it when I have time. This can be add to beta version relatively soon. But I'll have to wait for Meta to update rocksdb before releasing related features in non-beta versions, which can take much longer.

Congyuwang commented 2 months ago

Try pip install rocksdict==0.3.24b3.

Congyuwang commented 2 months ago

136

Congyuwang commented 2 months ago

Examples: https://github.com/Congyuwang/RocksDict/blob/main/examples/batch_write_wide_columns.py