ATAC-QC report no such file

yifeisun03 commented 5 years ago

Describe the bug Last step ATAQC get terminated and failed to generate QC report. The only file I see in this step's output directory is only regarding the gc report(trim.merged_gcPlot.pdf etc.) I suspect it's either picard fail to generate the output file or not erroring out on a missing input file. I'm still investigating this but any help be more than appreciated.

OS/Platform and dependencies

OS or Platform: Linux data2 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Cromwell/dxWDL version: cromwell-34.jar
singularity version: 3.0.1-1.el7.centos

Attach error logs

/usr/local/lib/python2.7/dist-packages/matplotlib-1.5.1-py2.7-linux-x86_64.egg/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache u\
sing fc-list. This may take a moment.
  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
Picked up _JAVA_OPTIONS: -Xms256M -Xmx15000M -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/sc/orga/scratch/suny04/pipeline-out/20190115/ATAC-Rat-Brow-Adipose_S5/cromwe\
ll-executions/atac/08e2be9b-6339-4048-bdaa-04c4b2d6c952/call-ataqc/shard-0/tmp.7e195714
Picked up _JAVA_OPTIONS: -Xms256M -Xmx15000M -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/sc/orga/scratch/suny04/pipeline-out/20190115/ATAC-Rat-Brow-Adipose_S5/cromwe\
ll-executions/atac/08e2be9b-6339-4048-bdaa-04c4b2d6c952/call-ataqc/shard-0/tmp.7e195714
Killed
Traceback (most recent call last):
  File "/software/atac-seq-pipeline/src/encode_ataqc.py", line 457, in <module>
    ataqc()
  File "/software/atac-seq-pipeline/src/encode_ataqc.py", line 149, in ataqc
    OUTPUT_PREFIX)
  File "/software/atac-seq-pipeline/src/run_ataqc.py", line 378, in get_picard_complexity_metrics
    with open(out_file, 'rb') as fp:
IOError: [Errno 2] No such file or directory: 'ATAC-Rat-Brow-Adipose_S5_L001_R1_001.trim.merged.picardcomplexity.qc'

leepc12 commented 5 years ago

Please make a tar ball of all logs (you can find an instruction for this when you create a new issue) and post it here.

yifeisun03 commented 5 years ago

debug_89.tar.gz

Attached is the tar.gz file, thank you very much!

leepc12 commented 5 years ago

It looks like the ATAQC job was killed by your cluster/computer (due to lack of memory? i guess). Please increase RAM >= 16GB on your instance/node/system and try again.

Picked up _JAVA_OPTIONS: -Xms256M -Xmx15000M -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/sc/orga/scratch/suny04/pipeline-out/20190115/ATAC-Rat-Brow-Adipose_S5/cromwe\
ll-executions/atac/08e2be9b-6339-4048-bdaa-04c4b2d6c952/call-ataqc/shard-0/tmp.7e195714
Killed

ENCODE-DCC / atac-seq-pipeline

ATAC-QC report no such file #89