ERROR: 2021-01-30 20:16:46,387 - Got exit code 1 (indicating failure) from job _toil_worker JobFunctionWrappingJob file:/mnt/storage/home/tk19812/scratch/HeliconiniiProject/HeliconGenomeAlignmentAnnotation/Test.ANN.pipeline.Cactus.CAT.Hmel.pb.cgp.outDir/toil/hints_db/Dpha/jobStore kind-JobFunctionWrappingJob/instance-7vr0fr0u.
WARNING: 2021-01-30 20:16:46,387 - Job failed with exit value 1: 'JobFunctionWrappingJob' kind-JobFunctionWrappingJob/instance-7vr0fr0u
WARNING: 2021-01-30 20:16:46,388 - The job seems to have left a log file, indicating failure: 'JobFunctionWrappingJob' kind-JobFunctionWrappingJob/instance-7vr0fr0u
WARNING: 2021-01-30 20:16:46,388 - Log from job kind-JobFunctionWrappingJob/instance-7vr0fr0u follows:
=========>
[2021-01-30T20:13:01+0000] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG---
[2021-01-30T20:13:01+0000] [MainThread] [I] [toil] Running Toil version 5.0.0-f182c6420554b258632a40bfa47a8f69e56675e4 on host compute121.bc4.acrc.priv.
[2021-01-30T20:13:01+0000] [MainThread] [I] [toil.worker] Working on job 'JobFunctionWrappingJob' kind-JobFunctionWrappingJob/instance-7vr0fr0u
[2021-01-30T20:13:01+0000] [MainThread] [I] [luigi-interface] Loaded ['luigi.cfg']
[2021-01-30T20:13:02+0000] [MainThread] [I] [toil.worker] Loaded body Job('JobFunctionWrappingJob' kind-JobFunctionWrappingJob/instance-7vr0fr0u) from description 'JobFunctionWrappingJob' kind-JobFunctionWrappingJob/instance-7vr0fr0u
^M processed line 1^M processed line 100001^M processed line 200001^M processed line 300001^M processed line 400001^M processed line 500001^M processed line 600001^M processed line 700001^M processed line 800001^M processed line 900001^M processed line 1000001^M processed line 1100001^M processed line 1200001^M processed line 1300001^M processed line 140
-------------------------------------
Processed alignments: 5146092
Summary of filtered alignments:
-------------------------------------
unmapped : 0
percent identity: 2991
coverage : 226076
not paired (our criterion) : 35944
quantiles of unspliced insert lengths:
q[10%]=127,q[20%]=151,q[30%]=181,q[40%]=236,q[50%]=432,q[60%]=699,q[70%]=1183,q[80%]=2049,q[90%]=6231,
not unique : 120733
singleton : 0
on diff. target : 0
-------------------------------------
Cmd line:
filterBam --uniq --in /mnt/storage/home/tk19812/scratch/HeliconiniiProject/HeliconGenomeAlignmentAnnotation/Test.ANN.pipeline.Cactus.CAT.Hmel.pb.cgp.outDir/node-799cb046-88c4-4b34-80da-a0657c1848f7-7f7f672c-9c5d-4a58-875f-dbdd25322867/tmpa4vz1gn2/68569b50-7c19-4d82-baad-597af64ddf26/tmpykgmca44.tmp --out /mnt/storage/scratch/tk19812/HeliconiniiProject/He
-------------------------------------
Elapsed time: 224 seconds.
ProcDag proc cleanup exception: an integer is required (got type NoneType)
[2021-01-30T20:16:46+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Failed job accessed files:
[2021-01-30T20:16:46+0000] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-JobFunctionWrappingJob/instance-br9cqc9f/file-6d19a825186f403cbe60ee42fd2b168b/compute121.bc4.acrc.priv.13991.8239473952.tmp' to path '/mnt/storage/home/tk19812/scratch/HeliconiniiProject/HeliconGenomeAlignmentAnnotation/Test.ANN.pipeline.C
Traceback (most recent call last):
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/venv/lib/python3.7/site-packages/toil/fileStores/nonCachingFileStore.py", line 63, in open
yield
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/venv/lib/python3.7/site-packages/toil/worker.py", line 393, in workerScript
job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer)
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/venv/lib/python3.7/site-packages/toil/job.py", line 2358, in _runner
returnValues = self._run(jobGraph=None, fileStore=fileStore)
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/venv/lib/python3.7/site-packages/toil/job.py", line 2279, in _run
return self.run(fileStore)
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/venv/lib/python3.7/site-packages/toil/job.py", line 2502, in run
rValue = userFunction(*((self,) + tuple(self._args)), **self._kwargs)
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/cat/hints_db.py", line 208, in filter_bam
tools.procOps.run_proc(sort_cmd)
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/tools/procOps.py", line 73, in run_proc
pl.wait()
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/tools/pipeline.py", line 1119, in wait
self.start()
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/tools/pipeline.py", line 1075, in start
self.__start()
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/tools/pipeline.py", line 1026, in __start
p._start()
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/tools/pipeline.py", line 716, in _start
self.raiseIfExcept()
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/tools/pipeline.py", line 749, in raiseIfExcept
raise self.exceptInfo[0].with_traceback(self.exceptInfo[2])
TypeError: descriptor 'with_traceback' requires a 'BaseException' object but received a 'traceback'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/venv/lib/python3.7/site-packages/toil/worker.py", line 393, in workerScript
job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer)
File "/mnt/storage/software/languages/anaconda/Anaconda3-2020.02-tflow-2.2.0/lib/python3.7/contextlib.py", line 130, in __exit__
self.gen.throw(type, value, traceback)
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/venv/lib/python3.7/site-packages/toil/fileStores/nonCachingFileStore.py", line 65, in open
diskUsed = getDirSizeRecursively(self.localTempDir)
File "/mnt/storage/scratch/tk19812/software/Comparative-Annotation-Toolkit/venv/lib/python3.7/site-packages/toil/common.py", line 1428, in getDirSizeRecursively
env=dict(os.environ, BLOCKSIZE='512')).decode('utf-8').split()[0]) * 512
File "/mnt/storage/software/languages/anaconda/Anaconda3-2020.02-tflow-2.2.0/lib/python3.7/subprocess.py", line 411, in check_output
**kwargs).stdout
File "/mnt/storage/software/languages/anaconda/Anaconda3-2020.02-tflow-2.2.0/lib/python3.7/subprocess.py", line 488, in run
with Popen(*popenargs, **kwargs) as process:
File "/mnt/storage/software/languages/anaconda/Anaconda3-2020.02-tflow-2.2.0/lib/python3.7/subprocess.py", line 800, in __init__
restore_signals, start_new_session)
File "/mnt/storage/software/languages/anaconda/Anaconda3-2020.02-tflow-2.2.0/lib/python3.7/subprocess.py", line 1482, in _execute_child
restore_signals, start_new_session, preexec_fn)
OSError: [Errno 12] Cannot allocate memory
[2021-01-30T20:16:46+0000] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host compute121.bc4.acrc.priv
report in: https://github.com/ComparativeGenomicsToolkit/Comparative-Annotation-Toolkit/issues/218#issuecomment-771714140