jamesls / semidbm

Cross platform (fast) DBM interface in python
Other
51 stars 13 forks source link

just a question about an issue i'm seeing with db disappearing #8

Open ghost opened 9 years ago

ghost commented 9 years ago

I have a dbm I use that I access via read-only sdbm handle in a web server, I never write to this handle (which wouldn't be possible anyway), but every once in awhile my data disappears, i.e. all the contents disappear (size goes from 2.6G to 8kb). It's hard to debug this because I don't write to the handle anywhere and it happens once every few weeks so it's hard for me to reproduce. I do access the read_only handle from multiple threads, so I'm wondering if somehow that's causing a problem? Here is how I create the handle

sdbm_read_only = sdbm.open(os.path.expandvars(self.config.get("sdbm_location")),'r')

ghost commented 9 years ago

and the threaded logic that accesses it

pool = gevent.pool.Pool(100) threads = [pool.spawn(self.read_from_sdbm,key) for key in things] pool.join()

jamesls commented 9 years ago

That's very concerning.

Is there any process on that box that ever writes to the db? How is the semidbm data file being initially populated?