search

stephan-hof / pyrocksdb

Python bindings for RocksDB
BSD 3-Clause "New" or "Revised" License
150 stars 169 forks source link

Memory leaks #20

Closed quard8 closed 9 years ago

quard8 commented 9 years ago

I tried to test pyrocksdb to store large cache data. I'm processing and write with batch about 10M keys with ~500 bytes of data.

Memory usage growth from 50MB to 900MB and after fixing possible leaks in my code, I did't see any changes.

After that, I tried plyvel just to check if I'm right. With plyvel and leveldb memory usage was 10-100MB.

Also, I can't use memory_profiler package, script never ends. Without memory_profile script usually works for 3-4 minutes.

pyrocksdb==0.3
rocksdb: stable 3.6.1 (bottled)
Python 2.7.8

So, can you check for possible memory leaks?

stephan-hof commented 9 years ago

Hmm is this script available ? Hunting a memory leak from scratch is not so easy.

stephan-hof commented 9 years ago

No response from the creator since a long time. As soon as he could submit a script to reproduce the issue I will reopen the ticket again.