bunbun
commented
8 years ago I did a quick google and found this: http://stackoverflow.com/questions/2740806/python-sqlite-database-is-locked
The checkpointing uses an SQLite database file (defaults to .ruffus_history.sqlite) that requires file locking semantics to ensure database integrity. The error message is telling you this has gone wrong. Here are some suggestions:
- Your underlying file system does not provide file locking semantics. The stack overflow article says this can fail if your directory is samba enabled....? Run sqlite manually on the checkpoint database $ sqlite3 .ruffus_history.sqlite
- Check you don't have a hung pipeline process sitting on the file. $ fuser .ruffus_history.sqlite
- Check you don't have multiple pipeline running instances using the same database file. (This is not sensible on so many levels...)
- Check there isn't a crashed session that has not been cleaned up properly $ ls .ruffus_history.sqlite-journal
- Check that the database is not corrupt $ sqlite3 .ruffus_history.sqlite "pragma integrity_check;" Try renaming the database $ mv .ruffus_history.sqlite .ruffus_history.sqlite.backup
Please tell us if any of this works. If your file system does not support placing the checkpoint database in your working directory, you can always put it elsewhere Leo
On 17 February 2016 at 05:43, AHCChan notifications@github.com wrote:
Hi,
I'm using a pipeline implemented by Ruffus that calls several other programs. I'm getting an error when Ruffus calls "bamM make".
This error only occurs when the data is stored on some parts of the server but not others. Any troubleshooting help would be greatly appreciated.
The error message is as follows:
Original exception:
Exception #1 '<class 'ruffus.ruffus_exceptions.RethrownJobError'>
Exceptions generating parameters for task = 'transcriptm.pipeline.QC_output'Original exception:
Exception #1 'sqlite3.OperationalError(database is locked)' raised in ... Task = def transcriptm.pipeline.QC_output(...): Traceback (most recent call last): File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/task.py", line 5118, in parameter_generator job_history, verbose_abbreviated_path): File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/task.py", line 4923, in job_needs_to_run verbose_abbreviated_path=verbose_abbreviated_path) File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/file_name_parameters.py", line 548, in needs_update_check_modify_time if os.path.relpath(p) not in job_history and p not in set_incomplete_files: File "/srv/sw/python/2.7.4/lib/python2.7/_abcoll.py", line 348, in __contains__ self[key] File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/dbdict.py", line 174, in __getitem__ (key, )).fetchone() OperationalError: database is locked ' raised in ...Traceback (most recent call last): File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/task.py", line 5887, in pipeline_run verbose) File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/task.py", line 5283, in fill_queue_with_job_parameters for params in job_parameters: File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/task.py", line 5215, in parameter_generator raise errt RethrownJobError:
Exceptions generating parameters for task = 'transcriptm.pipeline.QC_output'Original exception:
Exception #1 'sqlite3.OperationalError(database is locked)' raised in ... Task = def transcriptm.pipeline.QC_output(...): Traceback (most recent call last): File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/task.py", line 5118, in parameter_generator job_history, verbose_abbreviated_path): File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/task.py", line 4923, in job_needs_to_run verbose_abbreviated_path=verbose_abbreviated_path) File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/file_name_parameters.py", line 548, in needs_update_check_modify_time if os.path.relpath(p) not in job_history and p not in set_incomplete_files: File "/srv/sw/python/2.7.4/lib/python2.7/_abcoll.py", line 348, in __contains__ self[key] File "/srv/sw/ruffus/2.6.3/lib/python2.7/site-packages/ruffus/dbdict.py", line 174, in __getitem__ (key, )).fetchone() OperationalError: database is locked— Reply to this email directly or view it on GitHub https://github.com/bunbun/ruffus/issues/63.
AndreasHeger
Hi,
I'm using a pipeline implemented by Ruffus that calls several other programs. I'm getting an error when Ruffus calls "bamM make".
This error only occurs when the data is stored on some parts of the server but not others. Any troubleshooting help would be greatly appreciated.
The error message is as follows:
Original exception: