search

Congyuwang / RocksDict

Python fast on-disk dictionary / RocksDB & SpeeDB Python binding
https://congyuwang.github.io/RocksDict/rocksdict.html
MIT License
173 stars 8 forks source link

I get a "Too many open files" error #132

Closed orena1 closed 2 months ago

orena1 commented 2 months ago

Hi, I get this error when tryin to write to a file, I actually have many DB files open, one for each date, and I get: OSError: Too many open files (os error 24) I use WriteBatch and write it every 25k samples to accelerate the writetime

Congyuwang commented 2 months ago

I don't think it is a good idea to have MANY DBs. You just create one DB. encode your date information into your key. Say b'20240625-somekey'. Then you may want to use column families for different types of data, which also eliminates the need for multiple instances of DBs. Try this, it is more optimal.