search

ept / uploadr.py

Command-line Python script to upload photos to Flickr
Other
108 stars 56 forks source link

DB Error on uploading large sets of files #4

Open ankitdaf opened 11 years ago

ankitdaf commented 11 years ago

I am running a modified version of uploadr, and uploading a good sized photo collection, and I am getting the following error :

File "/media/C/Projects/uploadr/uploadr.py", line 419, in uploadImage if ( not self.uploaded.has_key( image ) ): File "/usr/lib/python2.7/shelve.py", line 107, in has_key return key in self.dict File "/usr/lib/python2.7/_abcoll.py", line 348, in contains self[key] File "/usr/lib/python2.7/bsddb/init.py", line 270, in getitem return _DeadlockWrap(lambda: self.db[key]) # self.db[key] File "/usr/lib/python2.7/bsddb/dbutils.py", line 68, in DeadlockWrap return function(args, *kwargs) File "/usr/lib/python2.7/bsddb/init.py", line 270, in return _DeadlockWrap(lambda: self.db[key]) # self.db[key] bsddb.db.DBRunRecoveryError: (-30973, 'DB_RUNRECOVERY: Fatal error, run database recovery -- PANIC: fatal region error detected; run recovery') Exception bsddb.db.DBRunRecoveryError: DBRunRecoveryError(-30973, 'DB_RUNRECOVERY: Fatal error, run database recovery -- PANIC: fatal region error detected; run recovery') in ignored Exception bsddb.db.DBRunRecoveryError: DBRunRecoveryError(-30973, 'DB_RUNRECOVERY: Fatal error, run database recovery -- PANIC: fatal region error detected; run recovery') in ignored

I suspect it is because the db is growing too big, since this error is thrown only after uploading a particularly large directory. Also, this error disappears for a while if I delete the "history" file, and appears a while later after sufficient number of pictures have been uploaded again.

ankitdaf commented 11 years ago

Just checked out the shelve module, I might be mistaken.

"When a program has a shelf open for writing, no other program should have it open for reading or writing."

Maybe this was the problem. Would be great to figure it out, one way or another.

ankitdaf commented 11 years ago

Alright, it definitely seems to be a problem with the size. The file wasn't opened, but problem persists after multiple uploads, and deleting the history file works.