search

kundajelab / atac_dnase_pipelines

ATAC-seq and DNase-seq processing pipeline
BSD 3-Clause "New" or "Revised" License
159 stars 81 forks source link

Can't kill a non-numeric process ID at /home/mycount/.bds/clusterGeneric/run.pl line 30 #102

Closed llz-hiv closed 6 years ago

llz-hiv commented 6 years ago

Hi, I met a issue when I run the pipeline. Below is the log, how can I fix it? Thank you!

li@ubuntu:~/atac_dnase_pipelines$ bds atac.bds -auto_detect_adapter -species hg19 -nth 6 -fastq1_1 SRR891280_1.fastq.gz -fastq1_2 SRR891280_2.fastq.gz Picked up _JAVA_OPTIONS: -Xms256M -Xmx728M -XX:ParallelGCThreads=1 ......... ......... == checking input files ...

Rep1 fastq (PE) : /home/li/atac_dnase_pipelines/SRR891280_1.fastq.gz /home/li/atac_dnase_pipelines/SRR891280_2.fastq.gz Distributing 6 to ... {1=6} Waiting for 58 seconds. 00:01:02.413 ExecutionerLocal 'Local[30]': Tasks [Local[30]] Pending: 1 Running: 1 Done: 0 Failed: 0 PID Task state Task name Dependencies Task definition
pending (SCHEDULED) detect_adapter rep1 python3 $(which detect_adapter.py) /home/li/atac_dnase_pipelines/SRR891280_2.fastq.gz > /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_2.adapter.txt; TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sle
1929 running (RUNNING) detect_adapter rep1 python3 $(which detect_adapter.py) /home/li/atac_dnase_pipelines/SRR891280_1.fastq.gz > /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.adapter.txt; TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sle
Waiting for 59 seconds. 00:02:03.835 ExecutionerLocal 'Local[30]': Tasks [Local[30]] Pending: 0 Running: 1 Done: 1 Failed: 0 PID Task state Task name Dependencies Task definition
1962 running (RUNNING) detect_adapter rep1 python3 $(which detect_adapter.py) /home/li/atac_dnase_pipelines/SRR891280_2.fastq.gz > /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_2.adapter.txt; TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sle

Detected adapter for rep1:00 (PE) : CTGTCTCTTATA, CTGTCTCTTATA Can't kill a non-numeric process ID at /home/li/.bds/clusterGeneric/run.pl line 30. Task failed: Program & line : 'modules/align_trim_adapter.bds', line 134 Task Name : 'trim_adapters_PE rep1' Task ID : 'atac.bds.20180416_113852_593_parallel_29/task.align_trim_adapter.trim_adapters_PE_rep1.line_134.id_12' Task PID : 'null' Task hint : 'cutadapt -m 5 -e 0.10 -a CTGTCTCTTATA -A CTGTCTCTTATA -o /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.fastq -p /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_2.trim.fastq /home/li/atac_dnase_pipelines/SRR891280_1.fastq.gz /home/li/atac_dnase_pipelines/SRR891280_2.fastq.gz; g' Task resources : 'cpus: -1 mem: 12.0 GB timeout: 82800 wall-timeout: 8640000' State : 'START_FAILED' Dependency state : 'ERROR' Retries available : '1' Input files : '[/home/li/atac_dnase_pipelines/SRR891280_1.fastq.gz, /home/li/atac_dnase_pipelines/SRR891280_2.fastq.gz]' Output files : '[/home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.fastq.gz, /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_2.trim.fastq.gz]' Script file : '/home/li/atac_dnase_pipelines/atac.bds.20180416_113852_593_parallel_29/task.align_trim_adapter.trim_adapters_PE_rep1.line_134.id_12.sh' Error message : 'Can't kill a non-numeric process ID at /home/li/.bds/clusterGeneric/run.pl line 30. ' Exit status : '1' Program : 

    # SYS command. line 136

     if [[ -f $(which conda) && $(conda env list | grep bds_atac | wc -l) != "0" ]]; then source activate bds_atac; sleep 5; fi;  export PATH=/home/li/atac_dnase_pipelines/.:/home/li/atac_dnase_pipelines/modules:/home/li/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)

    # SYS command. line 138

     cutadapt -m 5 -e 0.10 -a CTGTCTCTTATA -A CTGTCTCTTATA -o /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.fastq -p /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_2.trim.fastq /home/li/atac_dnase_pipelines/SRR891280_1.fastq.gz /home/li/atac_dnase_pipelines/SRR891280_2.fastq.gz

    # SYS command. line 140

     gzip -f /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.fastq

    # SYS command. line 141

     gzip -f /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_2.trim.fastq

    # SYS command. line 143

     TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi

Fatal error: atac.bds, line 767, pos 3. Task/s failed.

Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done. Fatal error: atac.bds, line 426, pos 2. Task/s failed.

Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.

leepc12 commented 6 years ago

Please see https://groups.google.com/forum/#!topic/klab_genomic_pipelines_discuss/jGRK2qFFO-U and update your pipeline and BDS.

Are you working on a SLURM cluster? Please run the following for debugging:

$ hostname -f
$ cat /home/li/.bds/clusterGeneric/run.pl
llz-hiv commented 6 years ago

Hi, thanks for your prompt reply. I already updated the BigDataScript and pipeline as well. But I am not working on a Slurm cluster. How can I run this pipline on my personal computer? Do I need to add the parameter '-q' when I run the pipeline?

leepc12 commented 6 years ago

No -q is for a system with cluster engines (SGE, SLURM).

I still want info for debugging:


$ hostname -f
$ cat /home/li/.bds/clusterGeneric/run.pl

On Mon, Apr 16, 2018 at 1:25 PM, llz-hiv notifications@github.com wrote:

Hi, thanks for your prompt reply. I already updated the BigDataScript and pipeline as well. But I am not working on a Slurm cluster. How can I run this pipline on my personal computer? Do I need to add the parameter '-q' when I run the pipeline?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kundajelab/atac_dnase_pipelines/issues/102#issuecomment-381737392, or mute the thread https://github.com/notifications/unsubscribe-auth/AIOd_LeUNeu-wiLkgKbdFLKrCfMWL8Zyks5tpP4igaJpZM4TXHKi .

llz-hiv commented 6 years ago

Below is the info for debugging:

li@ubuntu:~$ hostname -f ubuntu li@ubuntu:~$ cat /home/li/.bds/clusterGeneric/run.pl

!/usr/bin/perl

use POSIX;

die "Error: Missing arguments.\nUsage: run.pl timeout cpus mem queue saveStdout saveStderr cmd arg1 ... argN\n" if $#ARGV < 6 ;

$timeout = shift @ARGV; $cpus = shift @ARGV; $mem = shift @ARGV; $queue = shift @ARGV; $saveStdout = shift @ARGV; $saveStderr = shift @ARGV; $cmd = join(' ', @ARGV);

$qsub = "sbatch --export=ALL "; $qsub .= "-n 1 --ntasks-per-node=1 --cpus-per-task=$cpus " if( $cpus > 0 ); if( $mem > 0 ) { $mem = ceil($mem/1000000); # MB if ( $cpu > 0 ) { $mem = $mem*$cpu } $qsub .= "--mem $mem "; } if( $timeout > 0 ) { $timeout = ceil($timeout/60); # minute $qsub .= "-t $timeout "; } if ( $queue ne "" ) { $qsub .= " $queue " }

$pid = open QSUB, " | $qsub"; die "Cannot run command '$qsub'\n" if ! kill(0, $pid); # Check that process exists print QSUB "#!/bin/sh \n"; # SLURM sbatch needs this shebang... print QSUB "$cmd\n"; # Send cluster's task via qsub's STDIN close QSUB;

exit(0);

li@ubuntu:~$

leepc12 commented 6 years ago

Fixed. please get the latest pipeline and try again.

llz-hiv commented 6 years ago

It is done! Thanks! The pipeline is running, I will let you know if there is any problem.

leepc12 commented 6 years ago

First of all, i386 means 32-bit. Please remove any conflicting dependencies (like R, Anaconda, ...) in your ~/.bashrc or ~/.bash_profile.

Thanks,

Jin

On Tue, Apr 17, 2018 at 1:21 PM, llz-hiv notifications@github.com wrote:

PS: My system is 64-bit Linux Ubuntu 17.10. I tried to fix it by installing 32bit "multilib" version of libreadline by typing 'sudo apt-get install libreadline6:i386'. But it doesn't work.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kundajelab/atac_dnase_pipelines/issues/102#issuecomment-382129130, or mute the thread https://github.com/notifications/unsubscribe-auth/AIOd_NlAso9biqetvWSdHmAeMgD-qxbQks5tpk7GgaJpZM4TXHKi .

llz-hiv commented 6 years ago

Hi Jin, It seems that the workflow reaches the step of ATAqc. But there is no sufficient memory for JAVA environment. I have already add 'export TMPDIR=/home/li/tmp/' to .bashrc file, and add '-java_tmp_dir /home/li/javatmpdir/' in the command line. My Linux system has 8G memory, is it too small? Another issue: I use '-use_sambamba_markdup' in command line to avoid using picard, however, it still need the files (get_picard_complexity_metrics) created by picard when the pipline reach the ataqc step. How can I fix these problems? Below is the log:

ERROR 2018-04-17 19:42:50 ProcessExecutor Warning message: ERROR 2018-04-17 19:42:50 ProcessExecutor In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH, : ERROR 2018-04-17 19:42:50 ProcessExecutor zero-length arrow is of indeterminate angle and so skipped Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/home/li/javatmpdir/ 00:13:29.554 ExecutionerLocal 'Local[31]': Tasks [Local[31]] Pending: 0 Running: 1 Done: 6 Failed: 0 PID Task state Task name Dependencies Task definition
1929 running (RUNNING) ataqc rep1 if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then; fi; cd /home/li/atac_dnase_pipelines/out/qc/rep1; /home/li/atac_dnase_pipelines/ataqc/run_ataqc.py; --workdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outprefix SRR8912
00:14:33.586 ExecutionerLocal 'Local[31]': Tasks [Local[31]] Pending: 0 Running: 1 Done: 6 Failed: 0 PID Task state Task name Dependencies Task definition
1929 running (RUNNING) ataqc rep1 if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then; fi; cd /home/li/atac_dnase_pipelines/out/qc/rep1; /home/li/atac_dnase_pipelines/ataqc/run_ataqc.py; --workdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outprefix SRR8912

#

There is insufficient memory for the Java Runtime Environment to continue.

Native memory allocation (mmap) failed to map 756547584 bytes for committing reserved memory.

An error report file with more information is saved as:

/home/li/atac_dnase_pipelines/out/qc/rep1/hs_err_pid2072.log

OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000356200000, 756547584, 0) failed; error='Cannot allocate memory' (errno=12) Traceback (most recent call last): File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1654, in main() File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1450, in main OUTPUT_PREFIX) File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 371, in get_picard_complexity_metrics with open(out_file, 'rb') as fp: IOError: [Errno 2] No such file or directory: '/home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.picardcomplexity.qc' 00:15:31.632 ExecutionerLocal 'Local[31]': Tasks [Local[31]] Pending: 0 Running: 1 Done: 6 Failed: 0 PID Task state Task name Dependencies Task definition
1929 running (RUNNING) ataqc rep1 if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then; fi; cd /home/li/atac_dnase_pipelines/out/qc/rep1; /home/li/atac_dnase_pipelines/ataqc/run_ataqc.py; --workdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outprefix SRR8912

Task failed: Program & line : 'modules/ataqc.bds', line 109 Task Name : 'ataqc rep1' Task ID : 'atac.bds.20180417_193007_409_parallel_56/task.ataqc.ataqc_rep1.line_109.id_24' Task PID : '1929' Task hint : 'if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then; fi; cd /home/li/atac_dnase_pipelines/out/qc/rep1; /home/li/atac_dnase_pipelines/ataqc/run_ataqc.py; --workdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outprefix SRR8912' Task resources : 'cpus: -1 mem: -1.0 B wall-timeout: 8640000' State : 'ERROR' Dependency state : 'ERROR' Retries available : '1' Input files : '[/home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.fastq.gz, /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_2.trim.fastq.gz, /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.bam, /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.align.log, /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.nodup.pbc.qc, /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.dup.qc, /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.nodup.bam, /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.nodup.tn5.tagAlign.gz, /home/li/atac_dnase_pipelines/out/signal/macs2/rep1/SRR891280_1.trim.PE2SE.nodup.tn5.pf.pval.signal.bigwig, /home/li/atac_dnase_pipelines/out/peak/macs2/overlap/optimal_set/atac_dnase_pipelines_rep1-pr.naive_overlap.filt.narrowPeak.gz]' Output files : '[/home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE_qc.html, /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE_qc.txt]' Script file : '/home/li/atac_dnase_pipelines/atac.bds.20180417_193007_409_parallel_56/task.ataqc.ataqc_rep1.line_109.id_24.sh' Exit status : '1' Program : 

    # SYS command. line 111

     if [[ -f $(which conda) && $(conda env list | grep bds_atac | wc -l) != "0" ]]; then source activate bds_atac; sleep 5; fi;  export PATH=/home/li/atac_dnase_pipelines/.:/home/li/atac_dnase_pipelines/modules:/home/li/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)

    # SYS command. line 114

     export _JAVA_OPTIONS="-Xms256M -Xmx20G -XX:ParallelGCThreads=1"

    # SYS command. line 116

     if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then \
                export _JAVA_OPTIONS="${_JAVA_OPTIONS} -Djava.io.tmpdir=/home/li/javatmpdir/"; \
            fi

    # SYS command. line 119

     cd /home/li/atac_dnase_pipelines/out/qc/rep1

    # SYS command. line 122

     if [ -f "$(which picard)" ]; then export PICARDROOT="$(dirname $(which picard))/../share/picard"*; fi

    # SYS command. line 124

     /home/li/atac_dnase_pipelines/ataqc/run_ataqc.py \
                --workdir /home/li/atac_dnase_pipelines/out/qc/rep1 \
                --outdir /home/li/atac_dnase_pipelines/out/qc/rep1 \
                --outprefix SRR891280_1.trim.PE2SE \
                --genome hg19 \
                --chromsizes /home/li/atac_dnase_pipelines/DATA_DIR/hg19/hg19.chrom.sizes \
                --ref /home/li/atac_dnase_pipelines/DATA_DIR/hg19/male.hg19.fa \
                --tss /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/hg19_gencode_tss_unique.bed.gz \
                --dnase /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz \
                --blacklist /home/li/atac_dnase_pipelines/DATA_DIR/hg19/wgEncodeDacMapabilityConsensusExcludable.bed.gz \
                --prom /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/reg2map_honeybadger2_dnase_prom_p2.bed.gz \
                --enh /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/reg2map_honeybadger2_dnase_enh_p2.bed.gz \
                --pbc /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.nodup.pbc.qc\
                 --fastq1 /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.fastq.gz --fastq2 /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_2.trim.fastq.gz \
                --alignedbam /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.bam \
                --alignmentlog /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.align.log \
                --coordsortbam /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.bam \
                --duplog /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.dup.qc \
                --finalbam /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.nodup.bam \
                --finalbed /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.nodup.tn5.tagAlign.gz \
                --bigwig /home/li/atac_dnase_pipelines/out/signal/macs2/rep1/SRR891280_1.trim.PE2SE.nodup.tn5.pf.pval.signal.bigwig \
                --peaks /home/li/atac_dnase_pipelines/out/peak/macs2/overlap/optimal_set/atac_dnase_pipelines_rep1-pr.naive_overlap.filt.narrowPeak.gz  --naive_overlap_peaks /home/li/atac_dnase_pipelines/out/peak/macs2/overlap/optimal_set/atac_dnase_pipelines_rep1-pr.naive_overlap.filt.narrowPeak.gz   --use_sambamba_markdup \
                 --reg2map /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/dnase_avgs_reg2map_p10_merged_named.pvals.gz  --reg2map_bed /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz  --meta /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/eid_to_mnemonic.txt

    # SYS command. line 147

     rm -f test.log test.png

    # SYS command. line 149

     TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."; sleep 0
StdErr (100000000 lines)  :
    /home/li/miniconda3/envs/bds_atac/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
      warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
    Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/home/li/javatmpdir/
    ERROR   2018-04-17 19:42:50 ProcessExecutor Warning message:
    ERROR   2018-04-17 19:42:50 ProcessExecutor In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
    ERROR   2018-04-17 19:42:50 ProcessExecutor   zero-length arrow is of indeterminate angle and so skipped
    Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/home/li/javatmpdir/
    OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000356200000, 756547584, 0) failed; error='Cannot allocate memory' (errno=12)
    Traceback (most recent call last):
      File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1654, in <module>
        main()
      File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1450, in main
        OUTPUT_PREFIX)
      File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 371, in get_picard_complexity_metrics
        with open(out_file, 'rb') as fp:
    IOError: [Errno 2] No such file or directory: '/home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.picardcomplexity.qc'
StdOut (100000000 lines)  :
    #
    # There is insufficient memory for the Java Runtime Environment to continue.
    # Native memory allocation (mmap) failed to map 756547584 bytes for committing reserved memory.
    # An error report file with more information is saved as:
    # /home/li/atac_dnase_pipelines/out/qc/rep1/hs_err_pid2072.log

Fatal error: atac.bds, line 1616, pos 2. Task/s failed.

Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.

leepc12 commented 6 years ago

Yes, 8Gb is too small for ATAQC, but try adding -nth 1 to the command line and see what happens.

If it doesn't work then you can disable ATAQC with -no_ataqc.

Jin

On Tue, Apr 17, 2018 at 8:09 PM, llz-hiv notifications@github.com wrote:

Hi Jin, It seems that the workflow reaches the step of ATAqc. But there is no sufficient memory for JAVA environment. I have already add 'export TMPDIR=/home/li/tmp/' to .bashrc file, and add '-java_tmp_dir /home/li/javatmpdir/' in the command line. My Linux system has 8G memory, is it too small? Another issue: I use '-use_sambamba_markdup' in command line to avoid using picard, however, it still need the files (get_picard_complexity_metrics) created by picard when the pipline reach the ataqc step. How can I fix these problems? Below is the log:

ERROR 2018-04-17 19:42:50 ProcessExecutor Warning message: ERROR 2018-04-17 19:42:50 ProcessExecutor In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH, : ERROR 2018-04-17 19:42:50 ProcessExecutor zero-length arrow is of indeterminate angle and so skipped Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/home/li/javatmpdir/ 00:13:29.554 ExecutionerLocal 'Local[31]': Tasks [Local[31]] Pending: 0 Running: 1 Done: 6 Failed: 0 PID Task state Task name Dependencies Task definition

------------------------------------------------------------ | | 1929 | running (RUNNING) | ataqc rep1 | | if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then; fi; cd /home/li/atac_dnase_pipelines/out/qc/rep1; /home/li/atac_dnase_pipelines/ataqc/run_ataqc.py; --workdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outprefix SRR8912 |

00:14:33.586 ExecutionerLocal 'Local[31]': Tasks [Local[31]] Pending: 0 Running: 1 Done: 6 Failed: 0 PID Task state Task name Dependencies Task definition

------------------------------------------------------------ | | 1929 | running (RUNNING) | ataqc rep1 | | if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then; fi; cd /home/li/atac_dnase_pipelines/out/qc/rep1; /home/li/atac_dnase_pipelines/ataqc/run_ataqc.py; --workdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outprefix SRR8912 | There is insufficient memory for the Java Runtime Environment to continue. Native memory allocation (mmap) failed to map 756547584 bytes for committing reserved memory. An error report file with more information is saved as: /home/li/atac_dnase_pipelines/out/qc/rep1/hs_err_pid2072.log

OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000356200000, 756547584, 0) failed; error='Cannot allocate memory' (errno=12) Traceback (most recent call last): File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1654, in main() File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1450, in main OUTPUT_PREFIX) File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 371, in get_picard_complexity_metrics with open(out_file, 'rb') as fp: IOError: [Errno 2] No such file or directory: '/home/li/atacdnase pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.picardcomplexity.qc' 00:15:31.632 ExecutionerLocal 'Local[31]': Tasks [Local[31]] Pending: 0 Running: 1 Done: 6 Failed: 0 PID Task state Task name Dependencies Task definition

------------------------------------------------------------ | | 1929 | running (RUNNING) | ataqc rep1 | | if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then; fi; cd /home/li/atac_dnase_pipelines/out/qc/rep1; /home/li/atac_dnase_pipelines/ataqc/run_ataqc.py; --workdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outprefix SRR8912 |

Task failed: Program & line : 'modules/ataqc.bds', line 109 Task Name : 'ataqc rep1' Task ID : 'atac.bds.20180417_193007_409_parallel56/task.ataqc.ataqc rep1.line_109.id_24' Task PID : '1929' Task hint : 'if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then; fi; cd /home/li/atac_dnase_pipelines/out/qc/rep1; /home/li/atac_dnase_pipelines/ataqc/run_ataqc.py; --workdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outdir /home/li/atac_dnase_pipelines/out/qc/rep1; --outprefix SRR8912' Task resources : 'cpus: -1 mem: -1.0 B wall-timeout: 8640000' State : 'ERROR' Dependency state : 'ERROR' Retries available : '1' Input files : '[/home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.fastq.gz, /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_2.trim.fastq.gz, /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.bam, /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.align.log, /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.nodup.pbc.qc, /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.dup.qc, /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.nodup.bam, /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1. trim.PE2SE.nodup.tn5.tagAlign.gz, /home/li/atac_dnase_pipelines/ out/signal/macs2/rep1/SRR891280_1.trim.PE2SE.nodup.tn5.pf.pval.signal.bigwig, /home/li/atac_dnase_pipelines/out/peak/macs2/overlap/ optimal_set/atac_dnase_pipelinesrep1-pr.naive overlap.filt.narrowPeak.gz]' Output files : '[/home/li/atac_dnase_pipelines/out/qc/rep1/ SRR891280_1.trim.PE2SE_qc.html, /home/li/atac_dnase_pipelines/ out/qc/rep1/SRR891280_1.trim.PE2SE_qc.txt]' Script file : '/home/li/atac_dnasepipelines/atac.bds.20180417 193007_409_parallel_56/task.ataqc.ataqc_rep1.line_109.id_24.sh' Exit status : '1' Program :

SYS command. line 111

if [[ -f $(which conda) && $(conda env list | grep bds_atac | wc -l) != "0" ]]; then source activate bds_atac; sleep 5; fi; export PATH=/home/li/atac_dnase_pipelines/.:/home/li/atac_dnase_pipelines/modules:/home/li/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)

SYS command. line 114

export _JAVA_OPTIONS="-Xms256M -Xmx20G -XX:ParallelGCThreads=1"

SYS command. line 116

if [ "/home/li/javatmpdir/" != "" ] && [ -d "/home/li/javatmpdir/" ]; then \ export _JAVA_OPTIONS="${_JAVA_OPTIONS} -Djava.io.tmpdir=/home/li/javatmpdir/"; \ fi

SYS command. line 119

cd /home/li/atac_dnase_pipelines/out/qc/rep1

SYS command. line 122

if [ -f "$(which picard)" ]; then export PICARDROOT="$(dirname $(which picard))/../share/picard"*; fi

SYS command. line 124

/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py \ --workdir /home/li/atac_dnase_pipelines/out/qc/rep1 \ --outdir /home/li/atac_dnase_pipelines/out/qc/rep1 \ --outprefix SRR891280_1.trim.PE2SE \ --genome hg19 \ --chromsizes /home/li/atac_dnase_pipelines/DATA_DIR/hg19/hg19.chrom.sizes \ --ref /home/li/atac_dnase_pipelines/DATA_DIR/hg19/male.hg19.fa \ --tss /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/hg19_gencode_tss_unique.bed.gz \ --dnase /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz \ --blacklist /home/li/atac_dnase_pipelines/DATA_DIR/hg19/wgEncodeDacMapabilityConsensusExcludable.bed.gz \ --prom /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/reg2map_honeybadger2_dnase_prom_p2.bed.gz \ --enh /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/reg2map_honeybadger2_dnase_enh_p2.bed.gz \ --pbc /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.nodup.pbc.qc\ --fastq1 /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.fastq.gz --fastq2 /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_2.trim.fastq.gz \ --alignedbam /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.bam \ --alignmentlog /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.align.log \ --coordsortbam /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.bam \ --duplog /home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.dup.qc \ --finalbam /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.nodup.bam \ --finalbed /home/li/atac_dnase_pipelines/out/align/rep1/SRR891280_1.trim.PE2SE.nodup.tn5.tagAlign.gz \ --bigwig /home/li/atac_dnase_pipelines/out/signal/macs2/rep1/SRR891280_1.trim.PE2SE.nodup.tn5.pf.pval.signal.bigwig \ --peaks /home/li/atac_dnase_pipelines/out/peak/macs2/overlap/optimal_set/atac_dnase_pipelines_rep1-pr.naive_overlap.filt.narrowPeak.gz --naive_overlap_peaks /home/li/atac_dnase_pipelines/out/peak/macs2/overlap/optimal_set/atac_dnase_pipelines_rep1-pr.naive_overlap.filt.narrowPeak.gz --use_sambamba_markdup \ --reg2map /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/dnase_avgs_reg2map_p10_merged_named.pvals.gz --reg2map_bed /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz --meta /home/li/atac_dnase_pipelines/DATA_DIR/hg19/ataqc/eid_to_mnemonic.txt

SYS command. line 147

rm -f test.log test.png

SYS command. line 149

TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."; sleep 0 StdErr (100000000 lines) : /home/li/miniconda3/envs/bds_atac/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment. warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.') Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/home/li/javatmpdir/ ERROR 2018-04-17 19:42:50 ProcessExecutor Warning message: ERROR 2018-04-17 19:42:50 ProcessExecutor In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH, : ERROR 2018-04-17 19:42:50 ProcessExecutor zero-length arrow is of indeterminate angle and so skipped Picked up _JAVA_OPTIONS: -Xms256M -Xmx20G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/home/li/javatmpdir/ OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000356200000, 756547584, 0) failed; error='Cannot allocate memory' (errno=12) Traceback (most recent call last): File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1654, in main() File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1450, in main OUTPUT_PREFIX) File "/home/li/atac_dnase_pipelines/ataqc/run_ataqc.py", line 371, in get_picard_complexity_metrics with open(out_file, 'rb') as fp: IOError: [Errno 2] No such file or directory: '/home/li/atac_dnase_pipelines/out/qc/rep1/SRR891280_1.trim.PE2SE.picardcomplexity.qc' StdOut (100000000 lines) : #

There is insufficient memory for the Java Runtime Environment to continue.

Native memory allocation (mmap) failed to map 756547584 bytes for committing reserved memory.

An error report file with more information is saved as:

/home/li/atac_dnase_pipelines/out/qc/rep1/hs_err_pid2072.log

Fatal error: atac.bds, line 1616, pos 2. Task/s failed.

Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kundajelab/atac_dnase_pipelines/issues/102#issuecomment-382239100, or mute the thread https://github.com/notifications/unsubscribe-auth/AIOd_JTzEvJUDN4KEYS9Gm5x2wHXfJf1ks5tpq5zgaJpZM4TXHKi .

llz-hiv commented 6 years ago

Hi Jin, Thank you for your suggestion. I have already finished the pipeline without ATAqc. I have a naïve question: how can I visualize these results if no ataqc? I used firefox browser to open the report.html, and click the files showed in the directory structure, but all of them are text. How can I perform the subsequent analysis? I know little about bioinformatics. It will be greatly appreciated if you can kindly give me some suggestions.

Have a good day!

leepc12 commented 6 years ago

You need to keep all .png files and the /report directory along with the .html file. HTML file alone cannot show any images on it.

llz-hiv commented 6 years ago

Thank you very much!