search

BD2KGenomics / protect

Apache License 2.0
28 stars 13 forks source link

Fix "No Locks Available" Eror #69

Closed arkal closed 8 years ago

arkal commented 8 years ago 
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:  ---TOIL WORKER OUTPUT LOG---
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:  WARNING:toil.resource:The localize() method should only be invoked on a worker.
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:  WARNING:toil.resource:Can't find resource for leader path '/neoantigen_prediction/protect/src/protect'
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:  WARNING:toil.resource:Can't localize module ModuleDescriptor(dirPath='/neoantigen_prediction/protect/src', name='protect.pipeline.ProTECT')
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:  CRITICAL:toil.job:CACHE: Unable to acquire lock on /mnt/isilon/maris_lab/target_nbl_ngs/Eric/neoantigen_prediction/test_outputs/toil-64121748-44a6-4d46-bbb4-b0216cfdcd85/cache-64121748-44a6-4d46-bbb4-b0216cfdcd85/.cacheLock
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:  Traceback (most recent call last):
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:    File "/neoantigen_prediction/protect/venv/lib/python2.7/site-packages/toil/worker.py", line 334, in main
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:      with fileStore.open(job):
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:    File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:      return self.gen.next()
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:    File "/neoantigen_prediction/protect/venv/lib/python2.7/site-packages/toil/job.py", line 1054, in open
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,614 WARNING: toil.leader: C/w/jobE1qCG6:      with self._CacheState.open(self) as cacheInfo:
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:    File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:      return self.gen.next()
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:    File "/neoantigen_prediction/protect/venv/lib/python2.7/site-packages/toil/job.py", line 1826, in open
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:      with outer.cacheLock():
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:    File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:      return self.gen.next()
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:    File "/neoantigen_prediction/protect/venv/lib/python2.7/site-packages/toil/job.py", line 1433, in cacheLock
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:      flock(cacheLockFile, LOCK_EX)
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:  IOError: [Errno 37] No locks available
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:  Exiting the worker because of a failed jobWrapper on host reslnmarisig01.research.chop.edu
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:  ERROR:toil.worker:Exiting the worker because of a failed jobWrapper on host reslnmarisig01.research.chop.edu
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: C/w/jobE1qCG6:  WARNING:toil.jobWrapper:Due to failure we are reducing the remaining retry count of job C/w/jobE1qCG6 to 0
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,615 WARNING: toil.leader: Job: C/w/jobE1qCG6 is completely failed
reslnmarisig01.research.chop.edu: 2016-07-01 11:31:45,640 WARNING: toil.leader: The jobWrapper seems to have left a log file, indicating failure: n/i/jobTLYT2d

@hannes-ucsc any comments?

arkal commented 8 years ago

duplicates #70.