TypeError: cannot concatenate 'str' and 'NoneType' objects

shmilyfhh commented 5 years ago

Hi there,

I attached the error printout of one sample run. Any one could help? Thanks a lot.

Huihui

Target exists: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/QC_hg19/3_UTR New start mode: run anyway

Targetless command, running...

Traceback (most recent call last): File "/home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/pipelines/pepatac.py", line 1411, in sys.exit(main()) File "/home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/pipelines/pepatac.py", line 1287, in main container=pm.container) File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 716, in run self.callprint(cmd, nofail, container) # Run command File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 826, in callprint self._report_command(cmd) File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 1231, in _report_command print("\n> " + cmd + "\n") TypeError: cannot concatenate 'str' and 'NoneType' objects Pipeline status: running

Conditional flag found: []

These conditional files were left in place:['/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/fastq/2_3577UPitt_OM-MSC-1_ATAC_hs_i2.fastq', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/fastq/.fq', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/fastq/*.log', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/prealignments/tmp77IJ3D', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/prealignments/rCRSd_bt2', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/prealignments/tmpXeh9bk', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/prealignments/human_repeats_bt2', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19/tmpFxujBi', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19/2_3577UPitt_OM-MSC-1_ATAC_hs_i2_temp.bam.bai', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19/2_3577UPitt_OM-MSC-1_ATAC_hs_i2_temp.bam', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19/tmpP7Jl6D', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19_exact/temp/flag_completed', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19_exact/temp', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/peak_calling_hg19/2_3577UPitt_OM-MSC-1_ATAC_hs_i2_peaks.bed', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/peak_calling_hg19/chr_order.txt', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/peak_calling_hg19/2_3577UPitt_OM-MSC-1_ATAC_hs_i2_peaks_sort.bed'] Could not produce cleanup script for item(s):

/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/fastq/2_3577UPitt_OM-MSC-1_ATAC_hs_i2*.fastq
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/fastq/*.log
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/prealignments/tmp77IJ3D
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/prealignments/rCRSd_bt2
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/prealignments/tmpXeh9bk
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/prealignments/human_repeats_bt2
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19/tmpFxujBi
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19/2_3577UPitt_OM-MSC-1_ATAC_hs_i2_temp.bam.bai
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19/2_3577UPitt_OM-MSC-1_ATAC_hs_i2_temp.bam
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19/tmpP7Jl6D
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19_exact/temp/flag_completed
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/aligned_hg19_exact/temp
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/peak_calling_hg19/2_3577UPitt_OM-MSC-1_ATAC_hs_i2_peaks.bed
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/peak_calling_hg19/chr_order.txt
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/2_3577UPitt_OM-MSC-1_ATAC_hs_i2/peak_calling_hg19/2_3577UPitt_OM-MSC-1_ATAC_hs_i2_peaks_sort.bed

Pipeline failed at: (03-07 15:11:15) elapsed: 1.0 TIME

Total time: 1:49:49

Changed status from running to failed. Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 1632, in _exit_handler self.fail_pipeline(Exception("Pipeline failure. See details above.")) File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 1528, in fail_pipeline raise e Exception: Pipeline failure. See details above.

Pypiper terminating spawned child process 34088...(tee) Thu Mar 7 15:11:20 EST 2019 Done.

nsheff commented 5 years ago

that's strange. can you attach the whole output file by chance? I can't tell what the command is trying to do...

shmilyfhh commented 5 years ago

Thu Mar 7 11:31:38 EST 2019

[Pipeline run code and environment:]

Command: /home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/pipelines/pepatac.py -N -P 28 --single-or-paired paired --prealignments rCRSd human_repeats --genome hg19 --TSS-name hg19_TSS.tsv --sample-name 1_3577UPitt_A-MSC-1_ATAC_hs_i1 --input fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_r1.fastq --input2 fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_r2.fastq --genome-size hs --peak-caller macs2 --trimmer skewer --deduplicator samblaster -O pepATAC/
Compute host: node007
Working dir: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq
Outfolder: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/
Pipeline started at: (03-07 11:31:38) elapsed: 0.0 TIME

[Version log:]

Python version: 2.7.5
Pypiper dir: /primary/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper
Pypiper version: 0.9.4
Pipeline dir: /secondary/projects/shen/projects/tmp/piplines/pepatac/pipelines
Pipeline version: 0.8.4
Pipeline hash: fc48e9abab514f1fff4de04766d838853d3bfbf1
Pipeline branch: * master
Pipeline date: 2019-01-03 15:48:10 -0500
Pipeline diff: 1 file changed, 4 insertions(+), 4 deletions(-)

[Arguments passed to pipeline:]

input2: ['fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_r2.fastq']
paired_end: True
prealignments: ['rCRSd', 'human_repeats']
new_start: True
output_parent: pepATAC/
frip_ref_peaks: None
recover: False
TSS_name: hg19_TSS.tsv
deduplicator: samblaster
sample_name: 1_3577UPitt_A-MSC-1_ATAC_hs_i1
lite: False
input: ['fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_r1.fastq']
genome_size: hs
config_file: pepatac.yaml
mem: 4000
force_follow: False
cores: 28
anno_name: None
trimmer: skewer
peak_caller: macs2
keep: False
single_or_paired: paired
dirty: False
genome_assembly: hg19
no_fifo: False

Changed status from initializing to running.

Using custom config file: /home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/pipelines/pepatac.yaml

Loading config file: /home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/pipelines/pepatac.yaml

/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py:288: YAMLLoadWarning: Calling yaml.load() without Loader=... is deprecated. The default Loader is unsafe. *** Please read https://msg.pyyaml.org/load for full details. config = yaml.load(conf) Local input file: fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_r1.fastq Local input file: fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_r2.fastq

File_mb 9191.129 PEPATAC RES

Read_type paired PEPATAC RES

Genome hg19 PEPATAC RES

Merge/link and fastq conversion: (03-07 11:31:38) elapsed: 0.0 TIME

Number of input file sets: 2

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq

ln -sf /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_r1.fastq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq

Process 28555 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB) Local input file: '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq'

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.fastq

ln -sf /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_r2.fastq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.fastq

Process 28556 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB) Local input file: '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.fastq' Found .fastq file; no conversion necessary Found .fastq file; no conversion necessary

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq

ln -sf /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq

Process 28557 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB)

ln -sf /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.fastq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.fastq

Process 28558 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB) Follow: ['/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.fastq'] ['/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.fastq']

Raw_reads 66564988 PEPATAC RES

Fastq_reads 66564988 PEPATAC RES ['/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.fastq']

Adapter trimming: (03-07 11:31:53) elapsed: 15.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq

skewer -f sanger -t 28 -m pe -x /home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/tools/NexteraPE-PE.fa --quiet -o /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.fastq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.fastq

.--. .-.
: .--': :.-.
`. `. : `'.' .--. .-..-..-. .--. .--.
_`, :: . `.' '_.': `; `; :' '_.': ..'
`.__.':_;:_;`.__.'`.__.__.'`.__.':_;
skewer v0.2.2 [April 4, 2016]
Parameters used:
-- 3' end adapter sequences in file (-x):   /home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/tools/NexteraPE-PE.fa
A:  [0;33mAGATGTGTATAAGAGACAG
[0mB:  [0;33mAGATGTGTATAAGAGACAG
[0mC:  [0;33mTCGTCGGCAGCGTCAGATGTGTATAAGAGACAG
[0mD:  [0;33mCTGTCTCTTATACACATCTGACGCTGCCGACGA
[0mE:  [0;33mGTCTCGTGGGCTCGGAGATGTGTATAAGAGACAG
[0mF:  [0;33mCTGTCTCTTATACACATCTCCGAGCCCACGAGA
[0m-- maximum error ratio allowed (-r):    0.100
-- maximum indel error ratio allowed (-d):  0.030
-- minimum read length allowed after trimming (-l): 18
-- file format (-f):        Sanger/Illumina 1.8+ FASTQ 
-- number of concurrent threads (-t):   28
Thu Mar  7 11:31:53 2019[0;32m >> started[0m

Thu Mar  7 11:32:47 2019[0;32m >> done[0m (54.228s)
33282494 read pairs processed; of these:
      21 ( 0.00%) short read pairs filtered out after trimming by size control
      10 ( 0.00%) empty read pairs filtered out after trimming by size control
33282463 (100.00%) read pairs available; of these:
   59948 ( 0.18%) trimmed read pairs available after processing
33222515 (99.82%) untrimmed read pairs available after processing
log has been saved to "/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1-trimmed.log".

Process 28579 returned: (0). Elapsed: 0:00:54. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB)

mv /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1-trimmed-pair1.fastq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq

Process 28619 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB)

mv /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1-trimmed-pair2.fastq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq

Process 28620 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB) Follow: Evaluating read trimming

Trimmed_reads 66564926 PEPATAC RES

Trim_loss_rate 0.0 PEPATAC RES

Targetless command, running...

/primary/vari/software/fastqc/default/fastqc --noextract --outdir /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastqc /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq

Started analysis of 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 5% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 10% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 15% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 20% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 25% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 30% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 35% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 40% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 45% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 50% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 55% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 60% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 65% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 70% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 75% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 80% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 85% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 90% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Approx 95% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq
Analysis complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq

Process 28627 returned: (0). Elapsed: 0:01:38. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB)

FastQC report r1 fastqc/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim_fastqc.html FastQC report r1 None PEPATAC OBJ

Targetless command, running...

/primary/vari/software/fastqc/default/fastqc --noextract --outdir /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastqc /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq

Started analysis of 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 5% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 10% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 15% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 20% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 25% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 30% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 35% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 40% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 45% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 50% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 55% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 60% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 65% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 70% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 75% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 80% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 85% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 90% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Approx 95% complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq
Analysis complete for 1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq

Process 28668 returned: (0). Elapsed: 0:01:43. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB)

FastQC report r2 fastqc/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim_fastqc.html FastQC report r2 None PEPATAC OBJ

Prealignments (03-07 11:36:12) elapsed: 259.0 TIME

Prealignment assemblies: ['rCRSd', 'human_repeats']

Map to rCRSd (03-07 11:36:12) elapsed: 0.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/rCRSd_bt2

mkfifo /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/rCRSd_bt2

Process 28719 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 0.0GB)

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R2.fq

perl /home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/tools/filter_paired_fq.pl /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/rCRSd_bt2 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R2.trim.fastq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R1.fq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R2.fq

Not waiting for subprocess(es): [28720]

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_bt_aln_summary.log

(bowtie2 -p 28 -k 1 -D 20 -R 3 -N 1 -L 20 -i S,1,0.50 -x /home/huihui.fan/shen-secondary/projects/tmp/genomes/hg19/refgenie/rCRSd/indexed_bowtie2/rCRSd --rg-id 1_3577UPitt_A-MSC-1_ATAC_hs_i1 -U /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1_R1.trim.fastq --un /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/rCRSd_bt2 > /dev/null) 2>/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_bt_aln_summary.log

Process 28721 returned: (0). Elapsed: 0:07:33. Peak memory: (Process: 0.372GB; Pipeline: 0.372GB)

grep 'aligned exactly 1 time' /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_bt_aln_summary.log | awk '{print $1}'

Aligned_reads_rCRSd 23298870.0 PEPATAC RES

Alignment_rate_rCRSd 35.0 PEPATAC RES

Map to human_repeats (03-07 11:43:45) elapsed: 453.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/human_repeats_bt2

mkfifo /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/human_repeats_bt2

Process 28894 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 0.372GB)

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_unmap_R2.fq

perl /home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/tools/filter_paired_fq.pl /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/human_repeats_bt2 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R1.fq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R2.fq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_unmap_R1.fq /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_unmap_R2.fq

Not waiting for subprocess(es): [28895]

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_bt_aln_summary.log

(bowtie2 -p 28 -k 1 -D 20 -R 3 -N 1 -L 20 -i S,1,0.50 -x /home/huihui.fan/shen-secondary/projects/tmp/genomes/hg19/refgenie/human_repeats/indexed_bowtie2/human_repeats --rg-id 1_3577UPitt_A-MSC-1_ATAC_hs_i1 -U /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R1.fq --un /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/human_repeats_bt2 > /dev/null) 2>/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_bt_aln_summary.log

11649435 reads skipped
0 reads lost

Process 28896 returned: (0). Elapsed: 0:05:05. Peak memory: (Process: 0.405GB; Pipeline: 0.405GB)

grep 'aligned exactly 1 time' /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_bt_aln_summary.log | awk '{print $1}'

Aligned_reads_human_repeats 2534060.0 PEPATAC RES

Alignment_rate_human_repeats 3.81 PEPATAC RES

Map to genome (03-07 11:48:50) elapsed: 305.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam

bowtie2 -p 28 --very-sensitive -X 2000 --rg-id 1_3577UPitt_A-MSC-1_ATAC_hs_i1 -x /home/huihui.fan/shen-secondary/projects/tmp/genomes/hg19/refgenie/hg19/indexed_bowtie2/hg19 -1 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_unmap_R1.fq -2 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_unmap_R2.fq | samtools view -bS - -@ 1 | samtools sort - -@ 1 -T /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/tmpF6Im_7 -o /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam

1267030 reads skipped
0 reads lost
20365998 reads; of these:
  20365998 (100.00%) were paired; of these:
    572772 (2.81%) aligned concordantly 0 times
    17213026 (84.52%) aligned concordantly exactly 1 time
    2580200 (12.67%) aligned concordantly >1 times
    ----
    572772 pairs aligned concordantly 0 times; of these:
      94577 (16.51%) aligned discordantly 1 time
    ----
    478195 pairs aligned 0 times concordantly or discordantly; of these:
      956390 mates make up the pairs; of these:
        704019 (73.61%) aligned 0 times
        133818 (13.99%) aligned exactly 1 time
        118553 (12.40%) aligned >1 times
98.27% overall alignment rate

Process 29018 returned: (0). Elapsed: 0:18:04. Peak memory: (Process: 4.13GB; Pipeline: 4.13GB)

Process 29019 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 4.13GB)

[bam_sort_core] merging from 17 files...

Process 29020 returned: (0). Elapsed: 0:05:24. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

samtools view -q 10 -b -@ 28 -U /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_fail_qc.bam -f 2 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam > /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam

Process 29583 returned: (0). Elapsed: 0:01:20. Peak memory: (Process: 0.478GB; Pipeline: 4.13GB) Follow:

Mapped_reads 40027977 PEPATAC RES

QC_filtered_reads 2954949.0 PEPATAC RES

Aligned_reads 37073028 PEPATAC RES

Alignment_rate 55.69 PEPATAC RES

Total_efficiency 55.69 PEPATAC RES

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R1.fq.gz

gzip /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R1.fq

Process 29649 returned: (0). Elapsed: 0:04:28. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R2.fq.gz

gzip /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_rCRSd_unmap_R2.fq

Process 29765 returned: (0). Elapsed: 0:04:42. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_unmap_R1.fq.gz

gzip /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_unmap_R1.fq

Process 29851 returned: (0). Elapsed: 0:04:22. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_unmap_R2.fq.gz

gzip /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/1_3577UPitt_A-MSC-1_ATAC_hs_i1_human_repeats_unmap_R2.fq

Process 29932 returned: (0). Elapsed: 0:04:26. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam.bai

samtools index /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam

Process 30071 returned: (0). Elapsed: 0:00:25. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

samtools index /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam

Process 30076 returned: (0). Elapsed: 0:00:24. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

samtools idxstats /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam | grep -we 'chrM' -we 'chrMT' -we 'M' -we 'MT'| cut -f 3

Mitochondrial_reads 15517.0 PEPATAC RES

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_noMT.bam

samtools idxstats /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam | cut -f 1 | grep -vwe 'chrM' -vwe 'chrMT' -vwe 'M' -vwe 'MT'| xargs samtools view -b -@ 28 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam > /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_noMT.bam

Process 30090 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

Process 30091 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 4.13GB)

Process 30092 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 4.13GB)

Process 30093 returned: (0). Elapsed: 0:00:42. Peak memory: (Process: 0.486GB; Pipeline: 4.13GB)

mv /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_noMT.bam /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam

Process 30135 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

samtools index /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam

Process 30136 returned: (0). Elapsed: 0:00:24. Peak memory: (Process: 0.0GB; Pipeline: 4.13GB)

Calculate NRF, PBC1, and PBC2 (03-07 12:34:19) elapsed: 2730.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_bamQC.tsv

/home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/tools/bamQC.py -i /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam -c 28 -o /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_bamQC.tsv

Configured logger 'root' using pararead v0.5.0
Registering input file: '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam'
Temporary files will be stored in: '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/tmp_1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_DL7THA'
Processing with 28 cores...
[Name: chr19_gl000208_random; Size: 92689]
[Name: chr21_gl000210_random; Size: 27682]
[Name: chr6_apd_hap1; Size: 4622290]
[Name: chr13; Size: 115169878]
[Name: chr12; Size: 133851895]
[Name: chr11; Size: 135006516]
[Name: chr10; Size: 135534747]
[Name: chr17; Size: 81195210]
[Name: chr16; Size: 90354753]
[Name: chr15; Size: 102531392]
[Name: chr14; Size: 107349540]
[Name: chr19; Size: 59128983]
[Name: chr18; Size: 78077248]
[Name: chr9_gl000198_random; Size: 90085]
[Name: chrUn_gl000239; Size: 33824]
[Name: chrUn_gl000238; Size: 39939]
[Name: chrUn_gl000233; Size: 45941]
[Name: chrUn_gl000232; Size: 40652]
[Name: chrUn_gl000231; Size: 27386]
[Name: chrUn_gl000230; Size: 43691]
[Name: chrUn_gl000237; Size: 45867]
[Name: chrUn_gl000236; Size: 41934]
[Name: chrUn_gl000235; Size: 34474]
[Name: chrUn_gl000234; Size: 40531]
[Name: chr6_qbl_hap6; Size: 4611984]
[Name: chr11_gl000202_random; Size: 40103]
[Name: chr17_gl000206_random; Size: 41001]
[Name: chr6_cox_hap2; Size: 4795371]
[Name: chr4_gl000193_random; Size: 189789]
[Name: chrUn_gl000248; Size: 39786]
[Name: chrUn_gl000246; Size: 38154]
[Name: chrUn_gl000247; Size: 36422]
[Name: chrUn_gl000244; Size: 39929]
[Name: chrUn_gl000242; Size: 43523]
[Name: chrUn_gl000243; Size: 43341]
[Name: chrUn_gl000240; Size: 41933]
[Name: chrUn_gl000241; Size: 42152]
[Name: chr17_gl000204_random; Size: 81310]
[Name: chr17_ctg5_hap1; Size: 1680828]
[Name: chr17_gl000205_random; Size: 174588]
[Name: chr9_gl000199_random; Size: 169874]
[Name: chr9_gl000201_random; Size: 36148]
[Name: chr8; Size: 146364022]
[Name: chr6_ssto_hap7; Size: 4928567]
[Name: chr8_gl000197_random; Size: 37175]
[Name: chr6_dbb_hap3; Size: 4610396]
[Name: chr7_gl000195_random; Size: 182896]
[Name: chr1_gl000191_random; Size: 106433]
[Name: chr4_ctg9_hap1; Size: 590426]
[Name: chr3; Size: 198022430]
[Name: chrUn_gl000229; Size: 19913]
[Name: chr1; Size: 249250621]
[Name: chr17_gl000203_random; Size: 37498]
[Name: chr4_gl000194_random; Size: 191469]
[Name: chrY; Size: 59373566]
[Name: chrX; Size: 155270560]
[Name: chr9; Size: 141213431]
[Name: chr8_gl000196_random; Size: 38914]
[Name: chr6_mann_hap4; Size: 4683263]
[Name: chrUn_gl000211; Size: 166566]
[Name: chrUn_gl000213; Size: 164239]
[Name: chrUn_gl000212; Size: 186858]
[Name: chrUn_gl000215; Size: 172545]
[Name: chrUn_gl000214; Size: 137718]
[Name: chrUn_gl000217; Size: 172149]
[Name: chrUn_gl000216; Size: 172294]
[Name: chrUn_gl000219; Size: 179198]
[Name: chrUn_gl000218; Size: 161147]
[Name: chr19_gl000209_random; Size: 159169]
[Name: chr22; Size: 51304566]
[Name: chr20; Size: 63025520]
[Name: chr21; Size: 48129895]
[Name: chr6_mcf_hap5; Size: 4833398]
[Name: chr7; Size: 159138663]
[Name: chr6; Size: 171115067]
[Name: chr5; Size: 180915260]
[Name: chr4; Size: 191154276]
[Name: chrUn_gl000228; Size: 129120]
[Name: chr2; Size: 243199373]
[Name: chr1_gl000192_random; Size: 547496]
[Name: chrUn_gl000224; Size: 179693]
[Name: chrUn_gl000225; Size: 211173]
[Name: chrUn_gl000226; Size: 15008]
[Name: chrUn_gl000227; Size: 128374]
[Name: chrUn_gl000220; Size: 161802]
[Name: chrUn_gl000221; Size: 155397]
[Name: chrUn_gl000222; Size: 186861]
[Name: chrUn_gl000223; Size: 180455]
[Name: chr18_gl000207_random; Size: 4262]
Discarding 4 chunk(s) of reads: ['chrUn_gl000249', 'chrUn_gl000245', 'chr9_gl000200_random', 'chrM']
Keeping 89 chunk(s) of reads: ['chr19_gl000208_random', 'chr21_gl000210_random', 'chr6_apd_hap1', 'chr13', 'chr12', 'chr11', 'chr10', 'chr17', 'chr16', 'chr15', 'chr14', 'chr19', 'chr18', 'chr9_gl000198_random', 'chrUn_gl000239', 'chrUn_gl000238', 'chrUn_gl000233', 'chrUn_gl000232', 'chrUn_gl000231', 'chrUn_gl000230', 'chrUn_gl000237', 'chrUn_gl000236', 'chrUn_gl000235', 'chrUn_gl000234', 'chr6_qbl_hap6', 'chr11_gl000202_random', 'chr17_gl000206_random', 'chr6_cox_hap2', 'chr4_gl000193_random', 'chrUn_gl000248', 'chrUn_gl000246', 'chrUn_gl000247', 'chrUn_gl000244', 'chrUn_gl000242', 'chrUn_gl000243', 'chrUn_gl000240', 'chrUn_gl000241', 'chr17_gl000204_random', 'chr17_ctg5_hap1', 'chr17_gl000205_random', 'chr9_gl000199_random', 'chr9_gl000201_random', 'chr8', 'chr6_ssto_hap7', 'chr8_gl000197_random', 'chr6_dbb_hap3', 'chr7_gl000195_random', 'chr1_gl000191_random', 'chr4_ctg9_hap1', 'chr3', 'chrUn_gl000229', 'chr1', 'chr17_gl000203_random', 'chr4_gl000194_random', 'chrY', 'chrX', 'chr9', 'chr8_gl000196_random', 'chr6_mann_hap4', 'chrUn_gl000211', 'chrUn_gl000213', 'chrUn_gl000212', 'chrUn_gl000215', 'chrUn_gl000214', 'chrUn_gl000217', 'chrUn_gl000216', 'chrUn_gl000219', 'chrUn_gl000218', 'chr19_gl000209_random', 'chr22', 'chr20', 'chr21', 'chr6_mcf_hap5', 'chr7', 'chr6', 'chr5', 'chr4', 'chrUn_gl000228', 'chr2', 'chr1_gl000192_random', 'chrUn_gl000224', 'chrUn_gl000225', 'chrUn_gl000226', 'chrUn_gl000227', 'chrUn_gl000220', 'chrUn_gl000221', 'chrUn_gl000222', 'chrUn_gl000223', 'chr18_gl000207_random']
Reduce step (merge files)...
Merging 89 files into output file: '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_bamQC.tsv'

Process 30142 returned: (0). Elapsed: 0:00:25. Peak memory: (Process: 7.362GB; Pipeline: 7.362GB) Follow:

awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "NRF") c=i } getline; print $c }' /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_bamQC.tsv

awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "PBC1") c=i } getline; print $c }' /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_bamQC.tsv

awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "PBC2") c=i } getline; print $c }' /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_bamQC.tsv

NRF 0.93 PEPATAC RES

PBC1 0.96 PEPATAC RES

PBC2 26.51 PEPATAC RES

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_unmap.bam

samtools view -b -@ 28 -f 12 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam > /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_unmap.bam

Process 30187 returned: (0). Elapsed: 0:00:26. Peak memory: (Process: 0.031GB; Pipeline: 7.362GB) Follow:

samtools view -c -f 4 -@ 28 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam

Unmapped_reads 704019.0 PEPATAC RES

Remove dupes, build bigwig and bedgraph files (03-07 12:35:37) elapsed: 78.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_dedup.bam

samtools sort -n -@ 28 -T /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/tmpmb3Bu2 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort.bam | samtools view -h - -@ 28 | samblaster -r 2> /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_dedup_metrics_log.txt | samtools view -b - -@ 28 | samtools sort - -@ 28 -T /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/tmpmb3Bu2 -o /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_dedup.bam

Process 30232 returned: (0). Elapsed: 0:07:14. Peak memory: (Process: 0.0GB; Pipeline: 7.362GB)

Process 30233 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 7.362GB)

Process 30234 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 7.362GB)

Process 30235 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 7.362GB)

Process 30237 returned: (0). Elapsed: 0:01:00. Peak memory: (Process: 0.0GB; Pipeline: 7.362GB)

samtools index /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_dedup.bam

Process 30469 returned: (0). Elapsed: 0:00:24. Peak memory: (Process: 0.0GB; Pipeline: 7.362GB) Follow:

grep 'Removed' /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_dedup_metrics_log.txt | cut -f 3 -d ' '

Duplicate_reads 808827 PEPATAC RES

Dedup_aligned_reads 36264201.0 PEPATAC RES

Dedup_alignment_rate 54.48 PEPATAC RES

Dedup_total_efficiency 54.48 PEPATAC RES

Produce smoothed and nucleotide-resolution tracks (03-07 12:44:15) elapsed: 518.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/temp/flag_completed

/home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/tools/bamSitesToWig.py -i /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_dedup.bam -c /home/huihui.fan/shen-secondary/projects/tmp/genomes/hg19/refgenie/hg19/hg19.chromSizes -b /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/1_3577UPitt_A-MSC-1_ATAC_hs_i1_shift.bed -o /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/1_3577UPitt_A-MSC-1_ATAC_hs_i1_exact.bw -w /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_smooth.bw -p 18

Configured logger 'root' using pararead v0.5.0
Cutting parallel chroms in half to accommodate smooth track.
Registering input file: '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_dedup.bam'
Temporary files will be stored in: '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/tmp_1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_dedup_cuttrace_7aYgNe'
Processing with 9 cores...
[Name: chr19_gl000208_random; Size: 92689]
[Name: chr13; Size: 115169878]
[Name: chr10; Size: 135534747]
[Name: chr15; Size: 102531392]
[Name: chr18; Size: 78077248]
[Name: chrUn_gl000238; Size: 39939]
[Name: chrUn_gl000231; Size: 27386]
[Name: chrUn_gl000236; Size: 41934]
[Name: chr6_qbl_hap6; Size: 4611984]
[Name: chrUn_gl000230; Size: 43691]
[Name: chrUn_gl000233; Size: 45941]
[Name: chrUn_gl000235; Size: 34474]
[Name: chr21_gl000210_random; Size: 27682]
[Name: chrUn_gl000232; Size: 40652]
[Name: chrUn_gl000237; Size: 45867]
[Name: chrUn_gl000234; Size: 40531]
[Name: chr6_apd_hap1; Size: 4622290]
[Name: chr6_cox_hap2; Size: 4795371]
[Name: chrUn_gl000246; Size: 38154]
[Name: chrUn_gl000242; Size: 43523]
[Name: chrUn_gl000247; Size: 36422]
[Name: chrUn_gl000243; Size: 43341]
[Name: chrUn_gl000244; Size: 39929]
[Name: chrUn_gl000240; Size: 41933]
[Name: chrUn_gl000241; Size: 42152]
[Name: chr17_gl000205_random; Size: 174588]
[Name: chr17_gl000204_random; Size: 81310]
[Name: chr17_ctg5_hap1; Size: 1680828]
[Name: chr9_gl000199_random; Size: 169874]
[Name: chr9_gl000201_random; Size: 36148]
[Name: chr8; Size: 146364022]
[Name: chr6_dbb_hap3; Size: 4610396]
[Name: chr11_gl000202_random; Size: 40103]
[Name: chr17_gl000206_random; Size: 41001]
[Name: chr4_ctg9_hap1; Size: 590426]
[Name: chr1; Size: 249250621]
[Name: chr3; Size: 198022430]
[Name: chr4_gl000193_random; Size: 189789]
[Name: chrUn_gl000248; Size: 39786]
[Name: chrY; Size: 59373566]
[Name: chr7_gl000195_random; Size: 182896]
[Name: chr1_gl000191_random; Size: 106433]
[Name: chr8_gl000196_random; Size: 38914]
[Name: chr6_mann_hap4; Size: 4683263]
[Name: chrUn_gl000211; Size: 166566]
[Name: chrUn_gl000213; Size: 164239]
[Name: chrUn_gl000212; Size: 186858]
[Name: chrUn_gl000215; Size: 172545]
[Name: chrUn_gl000214; Size: 137718]
[Name: chrUn_gl000217; Size: 172149]
[Name: chrUn_gl000216; Size: 172294]
[Name: chrUn_gl000219; Size: 179198]
[Name: chrUn_gl000218; Size: 161147]
[Name: chr19_gl000209_random; Size: 159169]
[Name: chr22; Size: 51304566]
[Name: chrX; Size: 155270560]
[Name: chr20; Size: 63025520]
[Name: chr9_gl000198_random; Size: 90085]
[Name: chrUn_gl000239; Size: 33824]
[Name: chr6_mcf_hap5; Size: 4833398]
[Name: chr7; Size: 159138663]
[Name: chr14; Size: 107349540]
[Name: chr12; Size: 133851895]
[Name: chr21; Size: 48129895]
[Name: chr17; Size: 81195210]
[Name: chr6_ssto_hap7; Size: 4928567]
[Name: chr8_gl000197_random; Size: 37175]
[Name: chr5; Size: 180915260]
[Name: chr9; Size: 141213431]
[Name: chr2; Size: 243199373]
[Name: chr19; Size: 59128983]
[Name: chrUn_gl000229; Size: 19913]
[Name: chrUn_gl000225; Size: 211173]
[Name: chrUn_gl000226; Size: 15008]
[Name: chrUn_gl000227; Size: 128374]
[Name: chrUn_gl000220; Size: 161802]
[Name: chrUn_gl000221; Size: 155397]
[Name: chrUn_gl000222; Size: 186861]
[Name: chrUn_gl000223; Size: 180455]
[Name: chr18_gl000207_random; Size: 4262]
[Name: chr11; Size: 135006516]
[Name: chr16; Size: 90354753]
[Name: chr6; Size: 171115067]
[Name: chr17_gl000203_random; Size: 37498]
[Name: chr4_gl000194_random; Size: 191469]
[Name: chr4; Size: 191154276]
[Name: chr1_gl000192_random; Size: 547496]
[Name: chrUn_gl000224; Size: 179693]
[Name: chrUn_gl000228; Size: 129120]
Discarding 4 chunk(s) of reads: ['chrUn_gl000249', 'chrUn_gl000245', 'chr9_gl000200_random', 'chrM']
Keeping 89 chunk(s) of reads: ['chr19_gl000208_random', 'chr21_gl000210_random', 'chr6_apd_hap1', 'chr13', 'chr12', 'chr11', 'chr10', 'chr17', 'chr16', 'chr15', 'chr14', 'chr19', 'chr18', 'chr9_gl000198_random', 'chrUn_gl000239', 'chrUn_gl000238', 'chrUn_gl000233', 'chrUn_gl000232', 'chrUn_gl000231', 'chrUn_gl000230', 'chrUn_gl000237', 'chrUn_gl000236', 'chrUn_gl000235', 'chrUn_gl000234', 'chr6_qbl_hap6', 'chr11_gl000202_random', 'chr17_gl000206_random', 'chr6_cox_hap2', 'chr4_gl000193_random', 'chrUn_gl000248', 'chrUn_gl000246', 'chrUn_gl000247', 'chrUn_gl000244', 'chrUn_gl000242', 'chrUn_gl000243', 'chrUn_gl000240', 'chrUn_gl000241', 'chr17_gl000204_random', 'chr17_ctg5_hap1', 'chr17_gl000205_random', 'chr9_gl000199_random', 'chr9_gl000201_random', 'chr8', 'chr6_ssto_hap7', 'chr8_gl000197_random', 'chr6_dbb_hap3', 'chr7_gl000195_random', 'chr1_gl000191_random', 'chr4_ctg9_hap1', 'chr3', 'chrUn_gl000229', 'chr1', 'chr17_gl000203_random', 'chr4_gl000194_random', 'chrY', 'chrX', 'chr9', 'chr8_gl000196_random', 'chr6_mann_hap4', 'chrUn_gl000211', 'chrUn_gl000213', 'chrUn_gl000212', 'chrUn_gl000215', 'chrUn_gl000214', 'chrUn_gl000217', 'chrUn_gl000216', 'chrUn_gl000219', 'chrUn_gl000218', 'chr19_gl000209_random', 'chr22', 'chr20', 'chr21', 'chr6_mcf_hap5', 'chr7', 'chr6', 'chr5', 'chr4', 'chrUn_gl000228', 'chr2', 'chr1_gl000192_random', 'chrUn_gl000224', 'chrUn_gl000225', 'chrUn_gl000226', 'chrUn_gl000227', 'chrUn_gl000220', 'chrUn_gl000221', 'chrUn_gl000222', 'chrUn_gl000223', 'chr18_gl000207_random']
Reduce step (merge files)...
Merging 89 files into output file: '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/1_3577UPitt_A-MSC-1_ATAC_hs_i1_exact.bw'
Merging 89 files into output file: '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_smooth.bw'

Process 30480 returned: (0). Elapsed: 0:05:53. Peak memory: (Process: 14.606GB; Pipeline: 14.606GB)

touch /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/temp/flag_completed

Process 31412 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 14.606GB) Skipping TSS -- TSS enrichment requires TSS annotation file: /home/huihui.fan/shen-secondary/projects/tmp/genomes/hg19/refgenie/hg19/hg19_TSS.tsv

Call peaks (03-07 12:50:08) elapsed: 353.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.narrowPeak

/home/huihui.fan/.local/bin/macs2 callpeak -t /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/1_3577UPitt_A-MSC-1_ATAC_hs_i1_shift.bed -f BED -g hs --outdir /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19 -n 1_3577UPitt_A-MSC-1_ATAC_hs_i1 -q 0.01 --shift 0 --nomodel

INFO  @ Thu, 07 Mar 2019 12:50:08: 
# Command line: callpeak -t /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/1_3577UPitt_A-MSC-1_ATAC_hs_i1_shift.bed -f BED -g hs --outdir /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19 -n 1_3577UPitt_A-MSC-1_ATAC_hs_i1 -q 0.01 --shift 0 --nomodel
# ARGUMENTS LIST:
# name = 1_3577UPitt_A-MSC-1_ATAC_hs_i1
# format = BED
# ChIP-seq file = ['/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/1_3577UPitt_A-MSC-1_ATAC_hs_i1_shift.bed']
# control file = None
# effective genome size = 2.70e+09
# band width = 300
# model fold = [5, 50]
# qvalue cutoff = 1.00e-02
# The maximum gap between significant sites is assigned as the read length/tag size.
# The minimum length of peaks is assigned as the predicted fragment length "d".
# Larger dataset will be scaled towards smaller dataset.
# Range for calculating regional lambda is: 10000 bps
# Broad region calling is off
# Paired-End mode is off

INFO  @ Thu, 07 Mar 2019 12:50:08: #1 read tag files... 
INFO  @ Thu, 07 Mar 2019 12:50:08: #1 read treatment tags... 
INFO  @ Thu, 07 Mar 2019 12:50:09:  1000000 
INFO  @ Thu, 07 Mar 2019 12:50:10:  2000000 
INFO  @ Thu, 07 Mar 2019 12:50:11:  3000000 
INFO  @ Thu, 07 Mar 2019 12:50:13:  4000000 
INFO  @ Thu, 07 Mar 2019 12:50:14:  5000000 
INFO  @ Thu, 07 Mar 2019 12:50:15:  6000000 
INFO  @ Thu, 07 Mar 2019 12:50:16:  7000000 
INFO  @ Thu, 07 Mar 2019 12:50:17:  8000000 
INFO  @ Thu, 07 Mar 2019 12:50:18:  9000000 
INFO  @ Thu, 07 Mar 2019 12:50:20:  10000000 
INFO  @ Thu, 07 Mar 2019 12:50:21:  11000000 
INFO  @ Thu, 07 Mar 2019 12:50:22:  12000000 
INFO  @ Thu, 07 Mar 2019 12:50:23:  13000000 
INFO  @ Thu, 07 Mar 2019 12:50:24:  14000000 
INFO  @ Thu, 07 Mar 2019 12:50:25:  15000000 
INFO  @ Thu, 07 Mar 2019 12:50:26:  16000000 
INFO  @ Thu, 07 Mar 2019 12:50:27:  17000000 
INFO  @ Thu, 07 Mar 2019 12:50:28:  18000000 
INFO  @ Thu, 07 Mar 2019 12:50:29:  19000000 
INFO  @ Thu, 07 Mar 2019 12:50:30:  20000000 
INFO  @ Thu, 07 Mar 2019 12:50:31:  21000000 
INFO  @ Thu, 07 Mar 2019 12:50:32:  22000000 
INFO  @ Thu, 07 Mar 2019 12:50:33:  23000000 
INFO  @ Thu, 07 Mar 2019 12:50:34:  24000000 
INFO  @ Thu, 07 Mar 2019 12:50:35:  25000000 
INFO  @ Thu, 07 Mar 2019 12:50:36:  26000000 
INFO  @ Thu, 07 Mar 2019 12:50:37:  27000000 
INFO  @ Thu, 07 Mar 2019 12:50:38:  28000000 
INFO  @ Thu, 07 Mar 2019 12:50:39:  29000000 
INFO  @ Thu, 07 Mar 2019 12:50:40:  30000000 
INFO  @ Thu, 07 Mar 2019 12:50:41:  31000000 
INFO  @ Thu, 07 Mar 2019 12:50:42:  32000000 
INFO  @ Thu, 07 Mar 2019 12:50:43:  33000000 
INFO  @ Thu, 07 Mar 2019 12:50:44:  34000000 
INFO  @ Thu, 07 Mar 2019 12:50:45:  35000000 
INFO  @ Thu, 07 Mar 2019 12:50:46: #1 tag size is determined as 50 bps 
INFO  @ Thu, 07 Mar 2019 12:50:46: #1 tag size = 50.0 
INFO  @ Thu, 07 Mar 2019 12:50:46: #1  total tags in treatment: 35453413 
INFO  @ Thu, 07 Mar 2019 12:50:46: #1 user defined the maximum tags... 
INFO  @ Thu, 07 Mar 2019 12:50:46: #1 filter out redundant tags at the same location and the same strand by allowing at most 1 tag(s) 
INFO  @ Thu, 07 Mar 2019 12:50:46: #1  tags after filtering in treatment: 30887984 
INFO  @ Thu, 07 Mar 2019 12:50:46: #1  Redundant rate of treatment: 0.13 
INFO  @ Thu, 07 Mar 2019 12:50:46: #1 finished! 
INFO  @ Thu, 07 Mar 2019 12:50:46: #2 Build Peak Model... 
INFO  @ Thu, 07 Mar 2019 12:50:46: #2 Skipped... 
INFO  @ Thu, 07 Mar 2019 12:50:46: #2 Use 200 as fragment length 
INFO  @ Thu, 07 Mar 2019 12:50:46: #3 Call peaks... 
INFO  @ Thu, 07 Mar 2019 12:50:46: #3 Pre-compute pvalue-qvalue table... 
INFO  @ Thu, 07 Mar 2019 12:52:10: #3 Call peaks for each chromosome... 
INFO  @ Thu, 07 Mar 2019 12:53:15: #4 Write output xls file... /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.xls 
INFO  @ Thu, 07 Mar 2019 12:53:16: #4 Write peak in narrowPeak format file... /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.narrowPeak 
INFO  @ Thu, 07 Mar 2019 12:53:16: #4 Write summits bed file... /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_summits.bed 
INFO  @ Thu, 07 Mar 2019 12:53:17: Done!

Process 31413 returned: (0). Elapsed: 0:03:09. Peak memory: (Process: 0.0GB; Pipeline: 14.606GB) Follow:

Peak_count 126763 PEPATAC RES

Calculate fraction of reads in peaks (FRiP) (03-07 12:53:17) elapsed: 189.0 TIME

samtools view -@ 4 -c -L /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.narrowPeak /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_dedup.bam

('37073028', '12914714\n')

FRiP 0.348358758286 PEPATAC RES

Produce bigBed formatted narrowPeak file (03-07 12:54:07) elapsed: 50.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.bigBed

Rscript /home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/tools/narrowPeakToBigBed.R /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.narrowPeak /home/huihui.fan/shen-secondary/projects/tmp/genomes/hg19/refgenie/hg19/hg19.chromSizes bedToBigBed /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.bigBed

pass1 - making usageList (49 chroms): 53 millis
pass2 - checking and writing primary data (126764 records, 10 fields): 668 millis
[1] TRUE
[1] TRUE

Process 31488 returned: (0). Elapsed: 0:00:06. Peak memory: (Process: 0.0GB; Pipeline: 14.606GB)

Calculate peak coverage (03-07 12:54:12) elapsed: 6.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks_sort.bed

cut -f 1-3 /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.narrowPeak > /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.bed

Process 31522 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.003GB; Pipeline: 14.606GB)

samtools view -H /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_dedup.bam | grep 'SN:' | awk -F':' '{print $2,$3}' | awk -F' ' -v OFS=' ' '{print $1,$3}' > /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/chr_order.txt

Process 31524 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 14.606GB)

Process 31525 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 14.606GB)

Process 31526 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 14.606GB)

Process 31527 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 14.606GB)

bedtools sort -i /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.bed -faidx /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/chr_order.txt > /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks_sort.bed

Process 31529 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 14.606GB)

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks_coverage.bed

bedtools coverage -sorted -counts -a /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks_sort.bed -b /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_sort_dedup.bam -g /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/chr_order.txt > /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks_coverage.bed

Process 31531 returned: (0). Elapsed: 0:01:01. Peak memory: (Process: 0.008GB; Pipeline: 14.606GB)

Calculate read coverage (03-07 12:55:14) elapsed: 61.0 TIME

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/hg19_annotations.bed.gz

ln -sf /home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/anno/hg19_annotations.bed.gz /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/hg19_annotations.bed.gz

Process 31551 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: 0.0GB; Pipeline: 14.606GB)

gunzip -c /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/hg19_annotations.bed.gz | cut -f 4 | sort -u

Target to produce: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/3' UTR

gunzip -c /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/raw/hg19_annotations.bed.gz | awk -F' ' '{print>"/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/"$4}'

Process 31558 returned: (0). Elapsed: 0:00:01. Peak memory: (Process: 0.0GB; Pipeline: 14.606GB)

Process 31559 returned: (0). Elapsed: 0:00:00. Peak memory: (Process: None; Pipeline: 14.606GB)

Target exists: /secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/QC_hg19/3_UTR New start mode: run anyway

Targetless command, running...

Traceback (most recent call last): File "/home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/pipelines/pepatac.py", line 1411, in sys.exit(main()) File "/home/huihui.fan/shen-secondary/projects/tmp/piplines/pepatac/pipelines/pepatac.py", line 1287, in main container=pm.container) File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 716, in run self.callprint(cmd, nofail, container) # Run command File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 826, in callprint self._report_command(cmd) File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 1231, in _report_command print("\n> " + cmd + "\n") TypeError: cannot concatenate 'str' and 'NoneType' objects Pipeline status: running

Conditional flag found: []

These conditional files were left in place:['/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1.fastq', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/.fq', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/*.log', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/tmpjHOgSC', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/rCRSd_bt2', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/tmpxm8Dt3', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/human_repeats_bt2', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/tmpF6Im_7', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam.bai', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/tmpmb3Bu2', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/temp/flag_completed', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/temp', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.bed', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/chr_order.txt', '/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks_sort.bed'] Could not produce cleanup script for item(s):

/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/1_3577UPitt_A-MSC-1_ATAC_hs_i1*.fastq
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/fastq/*.log
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/tmpjHOgSC
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/rCRSd_bt2
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/tmpxm8Dt3
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/prealignments/human_repeats_bt2
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/tmpF6Im_7
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam.bai
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_temp.bam
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19/tmpmb3Bu2
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/temp/flag_completed
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/aligned_hg19_exact/temp
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks.bed
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/chr_order.txt
/secondary/projects/shen/projects/2019_01_07_MSC_Lan/ATACseq/pepATAC/1_3577UPitt_A-MSC-1_ATAC_hs_i1/peak_calling_hg19/1_3577UPitt_A-MSC-1_ATAC_hs_i1_peaks_sort.bed

Pipeline failed at: (03-07 12:55:15) elapsed: 1.0 TIME

Total time: 1:23:37

Changed status from running to failed. Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/usr/lib64/python2.7/atexit.py", line 24, in _run_exitfuncs func(*targs, **kargs) File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 1632, in _exit_handler self.fail_pipeline(Exception("Pipeline failure. See details above.")) File "/home/huihui.fan/.local/lib/python2.7/site-packages/piper-0.9.4-py2.7.egg/pypiper/manager.py", line 1528, in fail_pipeline raise e Exception: Pipeline failure. See details above.

Pypiper terminating spawned child process 28540...(tee) Child not responding to SIGTERM, trying SIGKILL...Thu Mar 7 12:55:20 EST 2019 Done.

nsheff commented 5 years ago

this is a bug in pepatac. You can get around it for now by turning off new start mode. you'll have to just erase your output manually for now, and then re-run.

we will work on solving the bug for the future, though.

shmilyfhh commented 5 years ago

sounds great! Thank you so much.

jpsmith5 commented 5 years ago

The most recent release includes the expected fix to this issue.

databio / pepatac