facebook / rocksdb

A library that provides an embeddable, persistent key-value store for fast storage.
http://rocksdb.org
GNU General Public License v2.0
27.9k stars 6.21k forks source link

what's the catch for avoid_unnecessary_blocking_io #12724

Closed zaidoon1 closed 1 month ago

zaidoon1 commented 1 month ago

I just discovered avoid_unnecessary_blocking_io and after reading the wiki https://github.com/facebook/rocksdb/wiki/IO#avoid-blocking-io as well as the code comments: https://github.com/facebook/rocksdb/blob/v9.2.1/include/rocksdb/options.h#L1316-L1322 It sounds like a win. However, it's not default on, why not? Is there a "catch"/caveats here?

ajkr commented 1 month ago

The main catch I can think of would be a problem when the high-pri thread pool is contended. Sending purges there could delay purge itself resulting in increased space/memory usage, or delay the other work that happens in the high-pri pool (memtable flushes). I doubt this would be a problem most of the time.

Using SstFileManager can move the actual deletion work to a dedicated thread. That could let you avoid adding I/O heavy work to the high-pri pool.

zaidoon1 commented 1 month ago

makes sense, thank you!