speedb-io / speedb

A RocksDB compliant high performance scalable embedded key-value store
https://www.speedb.io/
Apache License 2.0
912 stars 71 forks source link

[speedb rust binding] expose configurations not exposed by rust rocksdb binding #672

Open zaidoon1 opened 1 year ago

zaidoon1 commented 1 year ago

Currently, the speedb rust binding relies on the rust rocksdb rust binding (https://github.com/rust-rocksdb/rust-rocksdb) and therefore currently does not expose many configurations. The most important for me is the following:

  1. being able to set the logging destination as I don't have access to each individual machine that runs rocksdb, see: https://github.com/rust-rocksdb/rust-rocksdb/issues/814
  2. setting periodic compaction or compaction on delete. In my case, my db (compressed) fits in memory as it's small, however because it's small, It currently collects a lot of tombstones before triggering compaction which impacts read latency. The current workaround is to set small level sizes to try and trigger compaction often but this likely affects performance negatively as well.
  3. setting optimize filters for memory option: https://github.com/rust-rocksdb/rust-rocksdb/issues/807

This https://github.com/rust-rocksdb/rust-rocksdb/issues/260 also tracks the many other configurations that many users of the rust rocksdb binding are hoping to use so will likely help the community a lot if they were exposed: https://github.com/rust-rocksdb/rust-rocksdb/issues/260

zaidoon1 commented 1 year ago

https://github.com/speedb-io/rust-speedb/pull/1 adds some features listed here.

I have a pr (https://github.com/facebook/rocksdb/pull/12019) for rocksdb to expose period compaction seconds via c binding but it's not clear if/when I'll get approvals for it so I'll make one directly for speedb