twmht / python-rocksdb

Python bindings for RocksDB
BSD 3-Clause "New" or "Revised" License
276 stars 88 forks source link

segmentation fault #46

Closed iFA88 closed 4 years ago

iFA88 commented 5 years ago

Something went wrong when i delete the db pointer :) For me it is not a critical, because i never close the database without ending my program.

(fsPython) root@srv:/own/services/back/src/database/new# python
Python 3.5.3 (default, Sep 27 2018, 17:25:39) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rocksdb
>>> opt = rocksdb.Options()
>>> db = rocksdb.DB("/tmp/2323", rocksdb.Options(create_if_missing=True))
>>> db.close()
>>> del db
Segmentation fault
(fsPython) root@srv:/own/services/back/src/database/new# 

Maybe it is not good when i got this during exit:

(fsPython) root@srv:/own/services/back/src/database/new# python
Python 3.5.3 (default, Sep 27 2018, 17:25:39) 
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rocksdb
>>> opt = rocksdb.Options()
>>> db = rocksdb.DB("/tmp/2323", rocksdb.Options(create_if_missing=True))
>>> db.close()
>>> exit()
Segmentation fault
(fsPython) root@srv:/own/services/back/src/database/new# 

Linux srv 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19) x86_64 GNU/Linux

Update

Debug: https://gist.github.com/iFA88/1e688ea42a2e58d0f8ca88f16657acfa

itboyljm commented 4 years ago

hello, could you plz tell me what version of rocksdb you are using? Why there would be close method of Rocksdb DB obj?

iFA88 commented 4 years ago

Hey, sorry for the long inactivity. It was a very simple bug which has been fixed in my fork. The problem was, that during __dealloc__ started closing the already closed DB: https://github.com/iFA88/python-rocksdb/blob/master/rocksdb/_rocksdb.pyx#L1723-L1745

itboyljm commented 4 years ago

thank you so much for your fast reply. I try your version and it does solve the double free problem. But I still will meet the problem--"No Locks available" if I open, close and reopen, close same db object. Have you ever met this problem when you try to open after close same db?

iFA88 commented 4 years ago

Yes you have right, I will try to solve this problem.

update

It seems it wont work well.. /include/rocksdb/db.h:

// Close the DB by releasing resources, closing files etc. This should be
// called before calling the destructor so that the caller can get back a
// status in case there are any errors. This will not fsync the WAL files.
// If syncing is required, the caller must first call SyncWAL(), or Write()
// using an empty write batch with WriteOptions.sync=true.
// Regardless of the return status, the DB must be freed.
// If the return status is Aborted(), closing fails because there is
// unreleased snapshot in the system. In this case, users can release
// the unreleased snapshots and try again and expect it to succeed. For
// other status, recalling Close() will be no-op.
// If the return status is NotSupported(), then the DB implementation does
// cleanup in the destructor
virtual Status Close() { return Status::NotSupported(); }
binh-vu commented 3 years ago

I have the same issue when I close and reopen the db again. If I use the db.close() function I get segmentation fault. If I only run del db; gc.collect() then I get "No Locks available" like @itboyljm.

UPDATE:

I found that @iFA88 created a fork (https://github.com/iFA88/python-rocksdb) that fixed this issue. I put it out here in case anyone encounter the same problem. Thanks @iFA88