search

twmht / python-rocksdb

Python bindings for RocksDB
BSD 3-Clause "New" or "Revised" License
274 stars 89 forks source link

target_file_size_multiplier on pybind11 #64

Closed oersted closed 5 years ago

oersted commented 5 years ago

Since @twmht said that he would welcome requests for surfacing specific options in pybind11, here's one.

In my experience, it is important to increase the default target_file_size_multiplier = 1, I usually set it to 10, which is what the official Tuning Guide recommends. If target_file_size_multiplier = 1, all files regardless of the level have the same size. This means that the number of files increases linearly, and it is rather easy to hit the default Linux open file limit when operating at large scale. Setting it to 10 ensures that the number of files only grows logarithmically.

On a separate note, I am a bit confused about the options API, particularly about the difference between Options and DBOptions, can they be used in the same context? Btw, Options is shown in duplicate in the navigation bar of the docs.

twmht commented 5 years ago

@oersted

Options is inherited from DBOptions. According to rocksdb, some APIs with column families work with DBOptions.

twmht commented 5 years ago

please see https://github.com/twmht/python-rocksdb/blob/pybind11/tests/test_db.py#L32 for using target_file_size_multiplier.