Closed mkmwong closed 6 years ago
Thanks for reporting errors. Please make a tar ball of logs. You should be able to find the instruction for it when you create an issue.
Hi,
I think i found something - I had the same problem before when I ran the old version of pipeline. This is the stderr from call-macs2 folder:
[2018-10-11 01:09:02,147 INFO] ['/home/groups/ashbym/mandy/anaconda3/envs/encode-chip-seq-pipeline/bin/encode_macs2_chip.py', '/home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/a04b663b-6769-41ed-8f95-97c153ba38cb/call-macs2/shard-0/inputs/-209809900/SCGPM_RbWTK9K27_HTYFT_L7_TGACCA_R1.merged.nodup.tagAlign.gz', '/home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/a04b663b-6769-41ed-8f95-97c153ba38cb/call-macs2/shard-0/inputs/-686991457/ctl_for_rep1.tagAlign.gz', '--gensz', 'hs', '--chrsz', '/home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/a04b663b-6769-41ed-8f95-97c153ba38cb/call-macs2/shard-0/inputs/58336912/hg19.chrom.sizes', '--fraglen', '-5', '--cap-num-peak', '500000', '--pval-thresh', '0.01', '--make-signal', '--blacklist', '/home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/a04b663b-6769-41ed-8f95-97c153ba38cb/call-macs2/shard-0/inputs/58336912/wgEncodeDacMapabilityConsensusExcludable.bed.gz'] [2018-10-11 01:09:02,148 INFO] Initializing and making output directory... [2018-10-11 01:09:02,148 INFO] Calling peaks and generating signal tracks with MACS2... [2018-10-11 01:09:02,161 INFO] run_shell_cmd: PID=23840, CMD= macs2 callpeak -t /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/a04b663b-6769-41ed-8f95-97c153ba38cb/call-macs2/shard-0/inputs/-209809900/SCGPM_RbWTK9K27_HTYFT_L7_TGACCA_R1.merged.nodup.tagAlign.gz -c /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/a04b663b-6769-41ed-8f95-97c153ba38cb/call-macs2/shard-0/inputs/-686991457/ctl_for_rep1.tagAlign.gz -f BED -n SCGPM_RbWTK9K27_HTYFT_L7_TGACCA_R1.merged.nodup_x_ctl_for_rep1 -g hs -p 0.01 --nomodel --shift 0 --extsize -5 --keep-dup all -B --SPMR PID=23840: ERROR:root:--extsize must >= 1! [2018-10-11 01:09:04,219 ERROR] Unknown exception caught. Killing process group 23840... Traceback (most recent call last): File "/home/groups/ashbym/mandy/anaconda3/envs/encode-chip-seq-pipeline/bin/encode_common.py", line 224, in run_shell_cmd p.returncode, cmd) CalledProcessError: Command ' macs2 callpeak -t /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/a04b663b-6769-41ed-8f95-97c153ba38cb/call-macs2/shard-0/inputs/-209809900/SCGPM_RbWTK9K27_HTYFT_L7_TGACCA_R1.merged.nodup.tagAlign.gz -c /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/a04b663b-6769-41ed-8f95-97c153ba38cb/call-macs2/shard-0/inputs/-686991457/ctl_for_rep1.tagAlign.gz -f BED -n SCGPM_RbWTK9K27_HTYFT_L7_TGACCA_R1.merged.nodup_x_ctl_for_rep1 -g hs -p 0.01 --nomodel --shift 0 --extsize -5 --keep-dup all -B --SPMR' returned non-zero exit status 1
Is there anywhere we can manually change --extsize?
No, --extsize
is determined by a fragment length calculated from the cross-correlation analysis.
Please make a tar ball and attach it. I think this has something to do with the quality of your data.
Jin,
It will be good to have an option to over-ride ext-size. Because if the cross-correlation estimation of fragment length fails as in this case, the user should be allowed to set an --ext-size (and the --shift should be set accordingly).
-Anshul.
On Thu, Oct 11, 2018 at 1:54 PM Jin Lee notifications@github.com wrote:
No, --extsize is determined by a fragment length calculated from the cross-correlation analysis. Please make a tar ball and attach it. I think this has something to do with the quality of your data.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ENCODE-DCC/chip-seq-pipeline2/issues/31#issuecomment-429114966, or mute the thread https://github.com/notifications/unsubscribe-auth/AAI7Edp_-ACCpTfCYAQn4AZXdgp-gxM2ks5uj6_2gaJpZM4XYDor .
@akundaje: Sure, we will add it in the next release.
@akundaje: oh.. I forgot that it's already implemented in this pipeline.
@mkmwong: to manually specify --extsize
(fraglen) add chip.fraglen
to your input JSON. It's an integer array and the length of it should match with the number of replicates.
"chip.fraglen" : [100,100],
@akundaje: oh.. I forgot that it's already implemented in this pipeline.
@mkmwong: to manually specify
--extsize
(fraglen) addchip.fraglen
to your input JSON. It's an integer array and the length of it should match with the number of replicates."chip.fraglen" : [100,100],
Thank you Jin! One extra question, what would be a reasonable estimate for --ext-size? Would the average size of fragment be a good approximate for this?
100-150 should be work well.
On Thu, Oct 11, 2018, 6:05 PM Mandy Wong notifications@github.com wrote:
@akundaje https://github.com/akundaje: oh.. I forgot that it's already implemented in this pipeline.
@mkmwong https://github.com/mkmwong: to manually specify --extsize (fraglen) add chip.fraglen to your input JSON. It's an integer array and the length of it should match with the number of replicates.
"chip.fraglen" : [100,100],
Thank you Jin! One extra question, what would be a reasonable estimate for --ext-size? Would the average size of fragment be a good approximate for this?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ENCODE-DCC/chip-seq-pipeline2/issues/31#issuecomment-429170381, or mute the thread https://github.com/notifications/unsubscribe-auth/AAI7EZ5arLxcWpFwR83VorOOZAxi8p7Zks5uj-rygaJpZM4XYDor .
Hi,
I am not sure why after I used the fraglen setting, I encountered the following problem very early on in the pipeline: subprocess.CalledProcessError: Command 'samtools view -Su SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.merged.sam | samtools sort - SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.merged' returned non-zero exit s. I attached the tar ball here. Thanks! debug_31.tar.gz
Upon closer examination I've found a typo. I'm going to rerun the pipeline and see if thats the cause.
It's weird. Adding chip.fraglen
to the input JSON does not affect early steps of the pipeline. Did you change something else?
I don't think I changed anything else. I thought it was weird too. I rerun the pipeline after I corrected the typo and now it failed at a different step. I attached the tarball here. Please let me know if you have any leads on what's going on. Thanks! debug_31_new.tar.gz
Hi,
The closest I could find was this. where it suggested that the command was used to sort the sam file and covert it into a bam file is malformed.
2593 [2018-10-12 15:12:35,343 INFO] run_shell_cmd: PID=17948, CMD=samtools view -Su SCGPM_RbWTK9K27_HTYFT_L7_GATCAG_R1.merged.sam | samtools sort - SCGPM_R bWTK9K27_HTYFT_L7_GATCAG_R1.merged 2594 PID=17948: [bam_sort] Use -T PREFIX / -o FILE to specify temporary and final output files 2595 PID=17948: Usage: samtools sort [options...] [in.bam] 2596 PID=17948: Options: 2597 PID=17948: -l INT Set compression level, from 0 (uncompressed) to 9 (best) 2598 PID=17948: -m INT Set maximum memory per thread; suffix K/M/G recognized [768M] 2599 PID=17948: -n Sort by read name 2600 PID=17948: -t TAG Sort by value of TAG. Uses position as secondary index (or read name if -n is set) 2601 PID=17948: -o FILE Write final output to FILE rather than standard output 2602 PID=17948: -T PREFIX Write temporary files to PREFIX.nnnn.bam 2603 PID=17948: --input-fmt-option OPT[=VAL] 2604 PID=17948: Specify a single input file format option in the form 2605 PID=17948: of OPTION or OPTION=VALUE 2606 PID=17948: -O, --output-fmt FORMAT[,OPT[=VAL]]... 2607 PID=17948: Specify output format (SAM, BAM, CRAM) 2608 PID=17948: --output-fmt-option OPT[=VAL] 2609 PID=17948: Specify a single output file format option in the form 2610 PID=17948: of OPTION or OPTION=VALUE 2611 PID=17948: --reference FILE 2612 PID=17948: Reference sequence FASTA FILE [null] 2613 PID=17948: -@, --threads INT 2614 PID=17948: Number of additional threads to use [0] 2615 [2018-10-12 15:12:35,796 ERROR] Unknown exception caught. Killing process group 17948... 2616 Traceback (most recent call last): 2617 File "/home/groups/ashbym/mandy/anaconda3/envs/encode-chip-seq-pipeline/bin/encode_common.py", line 224, in run_shell_cmd 2618 p.returncode, cmd) 2619 subprocess.CalledProcessError: Command 'samtools view -Su SCGPM_RbWTK9K27_HTYFT_L7_GATCAG_R1.merged.sam | samtools sort - SCGPM_RbWTK9K27_HTYFT_L7_GAT CAG_R1.merged' returned non-zero exit status 1.
However, when I run the command directly, it runs just fine. Any idea on why that is? Thanks!
Can you run again without chip.fraglen
?
Can you run again without
chip.fraglen
?
Got the exact same error as the original post
It's very hard to figure out. Can you try with docker or singularity?
It's very hard to figure out. Can you try with docker or singularity?
I will try. The original error is from how the fraglen is estimated incorrectly. I just have one question about using chip.fraglen. Here's what's in my input file:
{ "chip.pipeline_type" : "histone", "chip.genome_tsv" : "/home/groups/ashbym/mandy/chip-seq-pipeline2/genome/hg19.tsv", "chip.fastqs_rep1_R1" : [ "/scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_TTAGGC_R1.fastq.gz" ], "chip.fastqs_rep1_R2" : [ "/scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_TTAGGC_R2.fastq.gz" ], "chip.fastqs_rep2_R1" : [ "/scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_TGACCA_R1.fastq.gz" ], "chip.fastqs_rep2_R2" : [ "/scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_TGACCA_R2.fastq.gz" ], "chip.ctl_fastqs_rep1_R1" : [ "/scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_ACAGTG_R1.fastq.gz" ], "chip.ctl_fastqs_rep1_R2" : [ "/scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_ACAGTG_R2.fastq.gz" ], "chip.ctl_fastqs_rep2_R1" : [ "/scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_GCCAAT_R1.fastq.gz" ], "chip.ctl_fastqs_rep2_R2" : [ "/scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_GCCAAT_R2.fastq.gz" ], "chip.paired_end" : true, "chip.no_jsd" : true, "chip.choose_ctl.always_use_pooled_ctl" : true, "chip.qc_report.name" : "WT_H3K27me3", "chip.qc_report.desc" : "H3K27me3 ChIP-Seq on WT IMR-90" }
when I use chip.fraglen, do I only use an array of length of 2? or do I need to use 8 because I have 8 files in total?
You have to use an array of length 2 [rep1, rep2]. You don't need fraglen for controls.
I have run this pipeline with deduplicated bam as input and that worked. However, when I ran with fastq.gz as input, I encounter following error:
[2018-10-10 15:34:55,23] [info] Running with database db.url = jdbc:hsqldb:mem:ecd30a73-9553-4e46-8424-5f4227afb55b;shutdown=false;hsqldb.tx=mvcc [2018-10-10 15:35:05,14] [info] Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000 [2018-10-10 15:35:05,16] [info] [RenameWorkflowOptionsInMetadata] 100% [2018-10-10 15:35:05,26] [info] Running with database db.url = jdbc:hsqldb:mem:1170c126-8f24-448b-99cb-8c3254c2f7dc;shutdown=false;hsqldb.tx=mvcc [2018-10-10 15:35:05,66] [[38;5;220mwarn[0m] This actor factory is deprecated. Please use cromwell.backend.google.pipelines.v1alpha2.PipelinesApiLifecycleActorFactory for PAPI v1 or cromwell.backend.google.pipelines.v2alpha1.PipelinesApiLifecycleActorFactory for PAPI v2 [2018-10-10 15:35:05,66] [[38;5;220mwarn[0m] Couldn't find a suitable DSN, defaulting to a Noop one. [2018-10-10 15:35:05,67] [info] Using noop to send events. [2018-10-10 15:35:05,97] [info] Slf4jLogger started [2018-10-10 15:35:06,27] [info] Workflow heartbeat configuration: { "cromwellId" : "cromid-bac7c2f", "heartbeatInterval" : "2 minutes", "ttl" : "10 minutes", "writeBatchSize" : 10000, "writeThreshold" : 10000 } [2018-10-10 15:35:06,42] [info] KvWriteActor configured to flush with batch size 200 and process rate 5 seconds. [2018-10-10 15:35:06,45] [info] Metadata summary refreshing every 2 seconds. [2018-10-10 15:35:06,47] [info] WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds. [2018-10-10 15:35:06,52] [info] CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds. [2018-10-10 15:35:07,62] [info] JobExecutionTokenDispenser - Distribution rate: 50 per 1 seconds. [2018-10-10 15:35:07,66] [info] SingleWorkflowRunnerActor: Version 34 [2018-10-10 15:35:07,68] [info] SingleWorkflowRunnerActor: Submitting workflow [2018-10-10 15:35:07,72] [info] PAPIQueryManager Running with 3 workers [2018-10-10 15:35:07,72] [info] JES batch polling interval is 33333 milliseconds [2018-10-10 15:35:07,72] [info] JES batch polling interval is 33333 milliseconds [2018-10-10 15:35:07,73] [info] JES batch polling interval is 33333 milliseconds [2018-10-10 15:35:07,82] [info] Unspecified type (Unspecified version) workflow f99b0390-cc7a-43a0-a7fa-294cf7d512af submitted [2018-10-10 15:35:07,88] [info] SingleWorkflowRunnerActor: Workflow submitted [38;5;2mf99b0390-cc7a-43a0-a7fa-294cf7d512af[0m [2018-10-10 15:35:07,88] [info] 1 new workflows fetched [2018-10-10 15:35:07,88] [info] WorkflowManagerActor Starting workflow [38;5;2mf99b0390-cc7a-43a0-a7fa-294cf7d512af[0m [2018-10-10 15:35:07,90] [info] WorkflowManagerActor Successfully started WorkflowActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [2018-10-10 15:35:07,90] [info] Retrieved 1 workflows from the WorkflowStoreActor [2018-10-10 15:35:07,91] [info] WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes. [2018-10-10 15:35:07,92] [[38;5;220mwarn[0m] SingleWorkflowRunnerActor: received unexpected message: Done in state RunningSwraData [2018-10-10 15:35:08,03] [info] MaterializeWorkflowDescriptorActor [[38;5;2mf99b0390[0m]: Parsing workflow as WDL draft-2 [2018-10-10 15:35:39,69] [info] MaterializeWorkflowDescriptorActor [[38;5;2mf99b0390[0m]: Call-to-Backend assignments: chip.overlap_ppr -> slurm, chip.bam2ta -> slurm, chip.qc_report -> slurm, chip.filter_ctl -> slurm, chip.idr_ppr -> slurm, chip.macs2_ppr2 -> slurm, chip.choose_ctl -> slurm, chip.fingerprint -> slurm, chip.xcor -> slurm, chip.spp_pooled -> slurm, chip.overlap_pr -> slurm, chip.spp_ppr1 -> slurm, chip.bam2ta_no_filt -> slurm, chip.macs2 -> slurm, chip.pool_ta_ctl -> slurm, chip.spp -> slurm, chip.merge_fastq_ctl -> slurm, chip.fraglen_mean -> slurm, chip.idr -> slurm, chip.pool_ta -> slurm, chip.overlap -> slurm, chip.read_genome_tsv -> slurm, chip.reproducibility_overlap -> slurm, chip.filter -> slurm, chip.macs2_pr1 -> slurm, chip.merge_fastq -> slurm, chip.idr_pr -> slurm, chip.spr -> slurm, chip.bam2ta_ctl -> slurm, chip.macs2_pr2 -> slurm, chip.macs2_ppr1 -> slurm, chip.macs2_pooled -> slurm, chip.pool_ta_pr2 -> slurm, chip.spp_pr1 -> slurm, chip.bwa -> slurm, chip.bam2ta_no_filt_R1 -> slurm, chip.spp_ppr2 -> slurm, chip.bwa_R1 -> slurm, chip.trim_fastq -> slurm, chip.bwa_ctl -> slurm, chip.pool_ta_pr1 -> slurm, chip.spp_pr2 -> slurm, chip.reproducibility_idr -> slurm [2018-10-10 15:35:39,84] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,85] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,86] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:39,87] [[38;5;220mwarn[0m] slurm [[38;5;2mf99b0390[0m]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. [2018-10-10 15:35:42,12] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.read_genome_tsv [2018-10-10 15:35:42,13] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition met: '!align_only && !true_rep_only'. Running conditional section [2018-10-10 15:35:42,13] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition met: '!true_rep_only'. Running conditional section [2018-10-10 15:35:42,74] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.read_genome_tsv:NA:1]: Unrecognized runtime attribute keys: disks [2018-10-10 15:35:43,20] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: 'enable_idr'. Bypassing conditional section [2018-10-10 15:35:43,22] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: 'peakcaller == "spp"'. Bypassing conditional section [2018-10-10 15:35:43,22] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition met: 'peakcaller == "macs2"'. Running conditional section [2018-10-10 15:35:43,22] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: '!align_only && !true_rep_only && enable_idr'. Bypassing conditional section [2018-10-10 15:35:43,22] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: 'enable_idr'. Bypassing conditional section [2018-10-10 15:35:43,22] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: 'peakcaller == "spp"'. Bypassing conditional section [2018-10-10 15:35:43,41] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.read_genome_tsv:NA:1]: [38;5;5mcat /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-read_genome_tsv/inputs/58336912/hg19.tsv[0m [2018-10-10 15:35:43,49] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.read_genome_tsv:NA:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_read_genome_tsv \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-read_genome_tsv \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-read_genome_tsv/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-read_genome_tsv/execution/stderr \ -t 60 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=4000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-read_genome_tsv/execution/script" [2018-10-10 15:35:46,30] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.merge_fastq_ctl, chip.merge_fastq [2018-10-10 15:35:46,47] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.read_genome_tsv:NA:1]: job id: 28445729 [2018-10-10 15:35:46,47] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.read_genome_tsv:NA:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 15:35:46,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq:0:1]: Unrecognized runtime attribute keys: disks [2018-10-10 15:35:46,65] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq_ctl:0:1]: Unrecognized runtime attribute keys: disks [2018-10-10 15:35:46,72] [[38;5;220mwarn[0m] Localization via hard link has failed: /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq_ctl/shard-0/inputs/661839578/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.fastq.gz -> /scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.fastq.gz: Invalid cross-device link [2018-10-10 15:35:46,72] [[38;5;220mwarn[0m] Localization via hard link has failed: /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq/shard-0/inputs/661839578/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.fastq.gz -> /scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.fastq.gz: Invalid cross-device link [2018-10-10 15:35:46,74] [[38;5;220mwarn[0m] Localization via hard link has failed: /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq/shard-0/inputs/661839578/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R2.fastq.gz -> /scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R2.fastq.gz: Invalid cross-device link [2018-10-10 15:35:46,74] [[38;5;220mwarn[0m] Localization via hard link has failed: /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq_ctl/shard-0/inputs/661839578/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R2.fastq.gz -> /scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R2.fastq.gz: Invalid cross-device link [2018-10-10 15:35:46,79] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq:0:1]: [38;5;5mpython $(which encode_merge_fastq.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq/shard-0/execution/write_tsv_82e931bc7d1a525bafe0ce1331b4b615.tmp \ --paired-end \ --nth 2[0m [2018-10-10 15:35:46,80] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq_ctl:0:1]: [38;5;5mpython $(which encode_merge_fastq.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq_ctl/shard-0/execution/write_tsv_5318eadf4f1e11ea3b195ca44b819fac.tmp \ --paired-end \ --nth 2[0m [2018-10-10 15:35:46,81] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_merge_fastq \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq/shard-0/execution/stderr \ -t 360 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=2 \ --mem=12000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq/shard-0/execution/script" [2018-10-10 15:35:46,83] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq_ctl:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_merge_fastq_ctl \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq_ctl/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq_ctl/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq_ctl/shard-0/execution/stderr \ -t 360 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=2 \ --mem=12000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-merge_fastq_ctl/shard-0/execution/script" [2018-10-10 15:35:51,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq:0:1]: job id: 28445734 [2018-10-10 15:35:51,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq_ctl:0:1]: job id: 28445732 [2018-10-10 15:35:51,47] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 15:35:51,47] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq_ctl:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 15:37:11,58] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 15:37:17,02] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.trim_fastq [2018-10-10 15:37:17,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.trim_fastq:0:1]: Unrecognized runtime attribute keys: disks [2018-10-10 15:37:17,69] [[38;5;220mwarn[0m] Localization via hard link has failed: /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-trim_fastq/shard-0/inputs/-567840873/merge_fastqs_R1_SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.fastq.gz -> /scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.fastq.gz: Invalid cross-device link [2018-10-10 15:37:17,70] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.trim_fastq:0:1]: [38;5;5mpython $(which encode_trim_fastq.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-trim_fastq/shard-0/inputs/-567840873/merge_fastqs_R1_SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.fastq.gz \ --trim-bp 50[0m [2018-10-10 15:37:17,74] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.trim_fastq:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_trim_fastq \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-trim_fastq/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-trim_fastq/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-trim_fastq/shard-0/execution/stderr \ -t 60 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=8000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-trim_fastq/shard-0/execution/script" [2018-10-10 15:37:21,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.trim_fastq:0:1]: job id: 28445811 [2018-10-10 15:37:21,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.trim_fastq:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 15:37:21,71] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.merge_fastq_ctl:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 15:37:42,59] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.read_genome_tsv:NA:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 15:37:45,57] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.bwa, chip.bwa_ctl [2018-10-10 15:37:45,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa:0:1]: Unrecognized runtime attribute keys: preemptible, disks [2018-10-10 15:37:45,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_ctl:0:1]: Unrecognized runtime attribute keys: preemptible, disks [2018-10-10 15:37:45,72] [[38;5;220mwarn[0m] Localization via hard link has failed: /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_ctl/shard-0/inputs/44672371/merge_fastqs_R1_SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.merged.fastq.gz -> /scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.fastq.gz: Invalid cross-device link [2018-10-10 15:37:45,73] [[38;5;220mwarn[0m] Localization via hard link has failed: /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa/shard-0/inputs/-567840873/merge_fastqs_R1_SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.fastq.gz -> /scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.fastq.gz: Invalid cross-device link [2018-10-10 15:37:45,74] [[38;5;220mwarn[0m] Localization via hard link has failed: /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_ctl/shard-0/inputs/44672371/merge_fastqs_R2_SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R2.merged.fastq.gz -> /scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R2.fastq.gz: Invalid cross-device link [2018-10-10 15:37:45,74] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_ctl:0:1]: [38;5;5mpython $(which encode_bwa.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_ctl/shard-0/inputs/-1093316416/male.hg19.fa.tar \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_ctl/shard-0/inputs/44672371/merge_fastqs_R1_SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.merged.fastq.gz /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_ctl/shard-0/inputs/44672371/merge_fastqs_R2_SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R2.merged.fastq.gz \ --paired-end \ --nth 4[0m [2018-10-10 15:37:45,75] [[38;5;220mwarn[0m] Localization via hard link has failed: /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa/shard-0/inputs/-567840873/merge_fastqs_R2_SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R2.merged.fastq.gz -> /scratch/users/mkmwong/RbKO_WT_ChIP/allFastq/original/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R2.fastq.gz: Invalid cross-device link [2018-10-10 15:37:45,76] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa:0:1]: [38;5;5mpython $(which encode_bwa.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa/shard-0/inputs/-1093316416/male.hg19.fa.tar \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa/shard-0/inputs/-567840873/merge_fastqs_R1_SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.fastq.gz /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa/shard-0/inputs/-567840873/merge_fastqs_R2_SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R2.merged.fastq.gz \ --paired-end \ --nth 4[0m [2018-10-10 15:37:45,78] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_ctl:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_bwa_ctl \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_ctl/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_ctl/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_ctl/shard-0/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=4 \ --mem=20000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_ctl/shard-0/execution/script" [2018-10-10 15:37:45,79] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_bwa \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa/shard-0/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=4 \ --mem=20000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa/shard-0/execution/script" [2018-10-10 15:37:51,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa:0:1]: job id: 28445834 [2018-10-10 15:37:51,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_ctl:0:1]: job id: 28445836 [2018-10-10 15:37:51,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_ctl:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 15:37:51,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 15:49:55,18] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.trim_fastq:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 15:50:00,71] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.bwa_R1 [2018-10-10 15:50:01,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_R1:0:1]: Unrecognized runtime attribute keys: preemptible, disks [2018-10-10 15:50:01,74] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_R1:0:1]: [38;5;5mpython $(which encode_bwa.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_R1/shard-0/inputs/-1093316416/male.hg19.fa.tar \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_R1/shard-0/inputs/913187384/merge_fastqs_R1_SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.trim_50bp.fastq.gz \ \ --nth 4[0m [2018-10-10 15:50:01,77] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_R1:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_bwa_R1 \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_R1/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_R1/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_R1/shard-0/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=4 \ --mem=20000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bwa_R1/shard-0/execution/script" [2018-10-10 15:50:06,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_R1:0:1]: job id: 28447299 [2018-10-10 15:50:06,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_R1:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 16:34:37,99] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_R1:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 16:34:40,06] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.bam2ta_no_filt_R1 [2018-10-10 16:34:40,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt_R1:0:1]: Unrecognized runtime attribute keys: disks [2018-10-10 16:34:40,71] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt_R1:0:1]: [38;5;5mpython $(which encode_bam2ta.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt_R1/shard-0/inputs/-55055628/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.trim_50bp.bam \ \ --disable-tn5-shift \ --regex-grep-v-ta 'chrM' \ --subsample 0 \[0m [2018-10-10 16:34:40,75] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt_R1:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_bam2ta_no_filt_R1 \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt_R1/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt_R1/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt_R1/shard-0/execution/stderr \ -t 360 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=2 \ --mem=10000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt_R1/shard-0/execution/script" [2018-10-10 16:34:41,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt_R1:0:1]: job id: 28451098 [2018-10-10 16:34:41,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt_R1:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 16:40:22,98] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt_R1:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 20:41:30,97] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 20:41:36,17] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.filter, chip.bam2ta_no_filt [2018-10-10 20:41:36,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt:0:1]: Unrecognized runtime attribute keys: disks [2018-10-10 20:41:36,69] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter:0:1]: Unrecognized runtime attribute keys: disks [2018-10-10 20:41:36,75] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter:0:1]: [38;5;5mpython $(which encode_filter.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter/shard-0/inputs/1119499152/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.bam \ --paired-end \ \ --dup-marker picard \ --mapq-thresh 30 \ \
ugly part to deal with optional outputs with Google JES backend
touch null[0m [2018-10-10 20:41:36,76] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt:0:1]: [38;5;5mpython $(which encode_bam2ta.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt/shard-0/inputs/1119499152/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.bam \ --paired-end \ --disable-tn5-shift \ --regex-grep-v-ta 'chrM' \ --subsample 0 \[0m [2018-10-10 20:41:36,79] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_bam2ta_no_filt \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt/shard-0/execution/stderr \ -t 360 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=2 \ --mem=10000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_no_filt/shard-0/execution/script" [2018-10-10 20:41:36,81] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_filter \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter/shard-0/execution/stderr \ -t 1440 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=2 \ --mem=20000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter/shard-0/execution/script" [2018-10-10 20:41:41,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt:0:1]: job id: 28469190 [2018-10-10 20:41:41,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter:0:1]: job id: 28469189 [2018-10-10 20:41:41,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 20:41:41,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 21:26:40,43] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_no_filt:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 22:34:18,23] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 22:34:23,63] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.bam2ta [2018-10-10 22:34:24,62] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta:0:1]: Unrecognized runtime attribute keys: disks [2018-10-10 22:34:24,70] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta:0:1]: [38;5;5mpython $(which encode_bam2ta.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta/shard-0/inputs/-1132106790/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.nodup.bam \ --paired-end \ --disable-tn5-shift \ --regex-grep-v-ta 'chrM' \ --subsample 0 \[0m [2018-10-10 22:34:24,73] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_bam2ta \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta/shard-0/execution/stderr \ -t 360 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=2 \ --mem=10000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta/shard-0/execution/script" [2018-10-10 22:34:26,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta:0:1]: job id: 28473079 [2018-10-10 22:34:26,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 23:00:31,64] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 23:00:34,78] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: '!true_reponly && length(tas) > 1 && peakcaller == "macs2"'. Bypassing conditional section [2018-10-10 23:00:34,78] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: '!true_reponly && length(tas) > 1'. Bypassing conditional section [2018-10-10 23:00:34,78] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: 'length(tas_) > 1 && peakcaller == "spp"'. Bypassing conditional section [2018-10-10 23:00:34,78] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: 'length(tas_) > 1'. Bypassing conditional section [2018-10-10 23:00:34,78] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: '!true_reponly && length(tas) > 1 && peakcaller == "spp"'. Bypassing conditional section [2018-10-10 23:00:34,78] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: '!true_reponly && length(tas) > 1 && peakcaller == "spp"'. Bypassing conditional section [2018-10-10 23:00:34,79] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: '!true_reponly && length(tas) > 1 && peakcaller == "macs2"'. Bypassing conditional section [2018-10-10 23:00:34,79] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: 'length(tas_) > 1'. Bypassing conditional section [2018-10-10 23:00:36,84] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.xcor, chip.spr [2018-10-10 23:00:37,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.spr:0:1]: Unrecognized runtime attribute keys: disks [2018-10-10 23:00:37,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.xcor:0:1]: Unrecognized runtime attribute keys: disks [2018-10-10 23:00:37,73] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.spr:0:1]: [38;5;5mpython $(which encode_spr.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-spr/shard-0/inputs/-922975244/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.nodup.tagAlign.gz \ --paired-end[0m [2018-10-10 23:00:37,73] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.xcor:0:1]: [38;5;5mpython $(which encode_xcor.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-xcor/shard-0/inputs/-155087896/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.trim_50bp.tagAlign.gz \ \ --subsample 15000000 \ --nth 2[0m [2018-10-10 23:00:37,76] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.spr:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_spr \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-spr/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-spr/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-spr/shard-0/execution/stderr \ -t 60 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=16000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-spr/shard-0/execution/script" [2018-10-10 23:00:37,77] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.xcor:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_xcor \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-xcor/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-xcor/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-xcor/shard-0/execution/stderr \ -t 360 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=2 \ --mem=16000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-xcor/shard-0/execution/script" [2018-10-10 23:00:41,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.xcor:0:1]: job id: 28475398 [2018-10-10 23:00:41,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.spr:0:1]: job id: 28475397 [2018-10-10 23:00:41,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.xcor:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 23:00:41,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.spr:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 23:06:01,11] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.spr:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 23:20:11,72] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.xcor:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-10 23:20:15,40] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.fraglen_mean [2018-10-10 23:20:15,63] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.fraglen_mean:NA:1]: Unrecognized runtime attribute keys: disks [2018-10-10 23:20:15,66] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.fraglenmean:NA:1]: [38;5;5mpython <<CODE arr = [-5] if len(arr): sum = sum(arr) mean = sum(arr)/float(len(arr)) print(int(round(mean))) else: print(0) CODE[0m [2018-10-10 23:20:15,69] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.fraglen_mean:NA:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_fraglen_mean \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-fraglen_mean \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-fraglen_mean/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-fraglen_mean/execution/stderr \ -t 60 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=4000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-fraglen_mean/execution/script" [2018-10-10 23:20:21,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.fraglen_mean:NA:1]: job id: 28476124 [2018-10-10 23:20:21,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.fraglen_mean:NA:1]: Status change from - to WaitingForReturnCodeFile [2018-10-10 23:22:22,12] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.fraglen_mean:NA:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-11 00:22:07,64] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bwa_ctl:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-11 00:22:13,31] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.filter_ctl [2018-10-11 00:22:13,63] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter_ctl:0:1]: Unrecognized runtime attribute keys: disks [2018-10-11 00:22:13,71] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter_ctl:0:1]: [38;5;5mpython $(which encode_filter.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter_ctl/shard-0/inputs/42388076/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.merged.bam \ --paired-end \ \ --dup-marker picard \ --mapq-thresh 30 \ \
ugly part to deal with optional outputs with Google JES backend
touch null[0m [2018-10-11 00:22:13,75] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter_ctl:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_filter_ctl \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter_ctl/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter_ctl/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter_ctl/shard-0/execution/stderr \ -t 1440 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=2 \ --mem=20000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-filter_ctl/shard-0/execution/script" [2018-10-11 00:22:16,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter_ctl:0:1]: job id: 28479287 [2018-10-11 00:22:16,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter_ctl:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-11 03:16:26,86] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.filter_ctl:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-11 03:16:30,91] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition NOT met: '!no_jsd && length(nodupbams) > 0 && length(ctl_nodupbams) > 0 && basename(blacklist) != "null"'. Bypassing conditional section [2018-10-11 03:16:32,95] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.bam2ta_ctl [2018-10-11 03:16:33,62] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_ctl:0:1]: Unrecognized runtime attribute keys: disks [2018-10-11 03:16:33,70] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_ctl:0:1]: [38;5;5mpython $(which encode_bam2ta.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_ctl/shard-0/inputs/-1207382602/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.merged.nodup.bam \ --paired-end \ --disable-tn5-shift \ --regex-grep-v-ta 'chrM' \ --subsample 0 \[0m [2018-10-11 03:16:33,73] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_ctl:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_bam2ta_ctl \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_ctl/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_ctl/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_ctl/shard-0/execution/stderr \ -t 360 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=2 \ --mem=10000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-bam2ta_ctl/shard-0/execution/script" [2018-10-11 03:16:36,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_ctl:0:1]: job id: 28483144 [2018-10-11 03:16:36,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2ta_ctl:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-11 03:47:39,16] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.bam2tactl:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-11 03:47:42,63] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition met: 'length(tas) > 0 && length(ctltas) > 0'. Running conditional section [2018-10-11 03:47:42,63] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Condition met: 'length(ctltas) > 0'. Running conditional section [2018-10-11 03:47:44,66] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.pool_ta_ctl [2018-10-11 03:47:45,62] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.pool_ta_ctl:NA:1]: Unrecognized runtime attribute keys: disks [2018-10-11 03:47:45,68] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.pool_ta_ctl:NA:1]: [38;5;5mpython $(which encode_pool_ta.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-pool_ta_ctl/inputs/1500631376/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.merged.nodup.tagAlign.gz[0m [2018-10-11 03:47:45,71] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.pool_ta_ctl:NA:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_pool_ta_ctl \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-pool_ta_ctl \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-pool_ta_ctl/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-pool_ta_ctl/execution/stderr \ -t 60 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=4000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-pool_ta_ctl/execution/script" [2018-10-11 03:47:46,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.pool_ta_ctl:NA:1]: job id: 28485165 [2018-10-11 03:47:46,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.pool_ta_ctl:NA:1]: Status change from - to WaitingForReturnCodeFile [2018-10-11 03:50:16,49] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.pool_ta_ctl:NA:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-11 03:50:18,63] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.choose_ctl [2018-10-11 03:50:19,62] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.choose_ctl:NA:1]: Unrecognized runtime attribute keys: disks [2018-10-11 03:50:19,74] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.choose_ctl:NA:1]: [38;5;5mpython $(which encode_choose_ctl.py) \ --tas /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-choose_ctl/inputs/-922975244/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.nodup.tagAlign.gz \ --ctl-tas /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-choose_ctl/inputs/1500631376/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.merged.nodup.tagAlign.gz \ \ --ctl-ta-pooled /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-choose_ctl/inputs/311943167/SCGPM_RbWTK9K27_HTYFT_L7_CTTGTA_R1.merged.nodup.tagAlign.gz \ --always-use-pooled-ctl \ --ctl-depth-ratio 1.2[0m [2018-10-11 03:50:19,78] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.choose_ctl:NA:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_choose_ctl \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-choose_ctl \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-choose_ctl/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-choose_ctl/execution/stderr \ -t 60 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=8000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-choose_ctl/execution/script" [2018-10-11 03:50:21,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.choose_ctl:NA:1]: job id: 28485274 [2018-10-11 03:50:21,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.choose_ctl:NA:1]: Status change from - to WaitingForReturnCodeFile [2018-10-11 03:52:47,47] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.choose_ctl:NA:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-11 03:52:51,57] [info] WorkflowExecutionActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af [[38;5;2mf99b0390[0m]: Starting chip.macs2, chip.macs2_pr2, chip.macs2_pr1 [2018-10-11 03:52:51,63] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2:0:1]: Unrecognized runtime attribute keys: disks [2018-10-11 03:52:51,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr2:0:1]: Unrecognized runtime attribute keys: disks [2018-10-11 03:52:51,64] [[38;5;220mwarn[0m] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr1:0:1]: Unrecognized runtime attribute keys: disks [2018-10-11 03:52:52,44] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2:0:1]: [38;5;5mpython $(which encode_macs2_chip.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2/shard-0/inputs/-922975244/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.nodup.tagAlign.gz /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2/shard-0/inputs/-1900987009/ctl_for_rep1.tagAlign.gz \ --gensz hs \ --chrsz /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2/shard-0/inputs/58336912/hg19.chrom.sizes \ --fraglen -5 \ --cap-num-peak 500000 \ --pval-thresh 0.01 \ --make-signal \ --blacklist /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2/shard-0/inputs/58336912/wgEncodeDacMapabilityConsensusExcludable.bed.gz
touch null # ugly part to deal with optional outputs[0m [2018-10-11 03:52:52,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr2:0:1]: [38;5;5mpython $(which encode_macs2_chip.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr2/shard-0/inputs/-303265875/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.nodup.pr2.tagAlign.gz /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr2/shard-0/inputs/-1900987009/ctl_for_rep1.tagAlign.gz \ --gensz hs \ --chrsz /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr2/shard-0/inputs/58336912/hg19.chrom.sizes \ --fraglen -5 \ --cap-num-peak 500000 \ --pval-thresh 0.01 \ \ --blacklist /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr2/shard-0/inputs/58336912/wgEncodeDacMapabilityConsensusExcludable.bed.gz
touch null.pval.signal.bigwig null.fc.signal.bigwig touch null # ugly part to deal with optional outputs[0m [2018-10-11 03:52:52,47] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr1:0:1]: [38;5;5mpython $(which encode_macs2_chip.py) \ /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr1/shard-0/inputs/1660355428/SCGPM_RbWTK9K27_HTYFT_L7_TAGCTT_R1.merged.nodup.pr1.tagAlign.gz /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr1/shard-0/inputs/-1900987009/ctl_for_rep1.tagAlign.gz \ --gensz hs \ --chrsz /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr1/shard-0/inputs/58336912/hg19.chrom.sizes \ --fraglen -5 \ --cap-num-peak 500000 \ --pval-thresh 0.01 \ \ --blacklist /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr1/shard-0/inputs/58336912/wgEncodeDacMapabilityConsensusExcludable.bed.gz
touch null.pval.signal.bigwig null.fc.signal.bigwig touch null # ugly part to deal with optional outputs[0m [2018-10-11 03:52:52,48] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_macs2 \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2/shard-0/execution/stderr \ -t 1440 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=16000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2/shard-0/execution/script" [2018-10-11 03:52:52,50] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr2:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_macs2_pr2 \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr2/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr2/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr2/shard-0/execution/stderr \ -t 1440 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=16000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr2/shard-0/execution/script" [2018-10-11 03:52:52,52] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr1:0:1]: executing: sbatch \ --export=ALL \ -J cromwell_f99b0390_macs2_pr1 \ -D /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr1/shard-0 \ -o /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr1/shard-0/execution/stdout \ -e /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr1/shard-0/execution/stderr \ -t 1440 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=16000 \ \ \ \ \ --wrap "/bin/bash /home/groups/ashbym/mandy/chip-seq-pipeline2/cromwell-executions/chip/f99b0390-cc7a-43a0-a7fa-294cf7d512af/call-macs2_pr1/shard-0/execution/script" [2018-10-11 03:52:56,45] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr1:0:1]: job id: 28485366 [2018-10-11 03:52:56,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr2:0:1]: job id: 28485368 [2018-10-11 03:52:56,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2:0:1]: job id: 28485367 [2018-10-11 03:52:56,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr1:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-11 03:52:56,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr2:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-11 03:52:56,46] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2:0:1]: Status change from - to WaitingForReturnCodeFile [2018-10-11 03:54:05,40] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr1:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-11 03:54:19,71] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-11 03:54:30,01] [info] DispatchedConfigAsyncJobExecutionActor [[38;5;2mf99b0390[0mchip.macs2_pr2:0:1]: Status change from WaitingForReturnCodeFile to Done [2018-10-11 03:54:30,47] [[38;5;1merror[0m] WorkflowManagerActor Workflow f99b0390-cc7a-43a0-a7fa-294cf7d512af failed (during ExecutingWorkflowState): cromwell.backend.standard.StandardAsyncExecutionActor$$anon$2: Failed to evaluate job outputs: Bad output 'macs2_pr1.npeak': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2_pr1.bfilt_npeak': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2_pr1.bfilt_npeak_bb': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2_pr1.frip_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$handleExecutionSuccess$1(StandardAsyncExecutionActor.scala:839) at scala.util.Success.$anonfun$map$1(Try.scala:251) at scala.util.Success.map(Try.scala:209) at scala.concurrent.Future.$anonfun$map$1(Future.scala:288) at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29) at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
cromwell.backend.standard.StandardAsyncExecutionActor$$anon$2: Failed to evaluate job outputs: Bad output 'macs2.npeak': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2.bfilt_npeak': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2.bfilt_npeak_bb': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2.sig_pval': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2.sig_fc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2.frip_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$handleExecutionSuccess$1(StandardAsyncExecutionActor.scala:839) at scala.util.Success.$anonfun$map$1(Try.scala:251) at scala.util.Success.map(Try.scala:209) at scala.concurrent.Future.$anonfun$map$1(Future.scala:288) at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29) at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
cromwell.backend.standard.StandardAsyncExecutionActor$$anon$2: Failed to evaluate job outputs: Bad output 'macs2_pr2.npeak': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2_pr2.bfilt_npeak': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2_pr2.bfilt_npeak_bb': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 Bad output 'macs2_pr2.frip_qc': Failed to find index Success(WomInteger(0)) on array:
Success([])
0 at cromwell.backend.standard.StandardAsyncExecutionActor.$anonfun$handleExecutionSuccess$1(StandardAsyncExecutionActor.scala:839) at scala.util.Success.$anonfun$map$1(Try.scala:251) at scala.util.Success.map(Try.scala:209) at scala.concurrent.Future.$anonfun$map$1(Future.scala:288) at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29) at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29) at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[2018-10-11 03:54:30,48] [info] WorkflowManagerActor WorkflowActor-f99b0390-cc7a-43a0-a7fa-294cf7d512af is in a terminal state: WorkflowFailedState [2018-10-11 03:54:32,98] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'. [2018-10-11 03:54:36,49] [info] Workflow polling stopped [2018-10-11 03:54:36,50] [info] Shutting down WorkflowStoreActor - Timeout = 5 seconds [2018-10-11 03:54:36,51] [info] Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds [2018-10-11 03:54:36,51] [info] Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds [2018-10-11 03:54:36,51] [info] JobExecutionTokenDispenser stopped [2018-10-11 03:54:36,51] [info] Aborting all running workflows. [2018-10-11 03:54:36,51] [info] WorkflowStoreActor stopped [2018-10-11 03:54:36,52] [info] Shutting down WorkflowManagerActor - Timeout = 3600 seconds [2018-10-11 03:54:36,52] [info] WorkflowLogCopyRouter stopped [2018-10-11 03:54:36,52] [info] Connection pools shut down [2018-10-11 03:54:36,52] [info] Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds [2018-10-11 03:54:36,52] [info] Shutting down JobStoreActor - Timeout = 1800 seconds [2018-10-11 03:54:36,52] [info] Shutting down CallCacheWriteActor - Timeout = 1800 seconds [2018-10-11 03:54:36,52] [info] Shutting down ServiceRegistryActor - Timeout = 1800 seconds [2018-10-11 03:54:36,52] [info] Shutting down DockerHashActor - Timeout = 1800 seconds [2018-10-11 03:54:36,52] [info] Shutting down IoProxy - Timeout = 1800 seconds [2018-10-11 03:54:36,52] [info] WorkflowManagerActor stopped [2018-10-11 03:54:36,52] [info] DockerHashActor stopped [2018-10-11 03:54:36,52] [info] IoProxy stopped [2018-10-11 03:54:36,52] [info] CallCacheWriteActor Shutting down: 0 queued messages to process [2018-10-11 03:54:36,52] [info] SubWorkflowStoreActor stopped [2018-10-11 03:54:36,52] [info] CallCacheWriteActor stopped [2018-10-11 03:54:36,52] [info] WorkflowManagerActor All workflows finished [2018-10-11 03:54:36,52] [info] JobStoreActor stopped [2018-10-11 03:54:36,53] [info] WriteMetadataActor Shutting down: 0 queued messages to process [2018-10-11 03:54:36,53] [info] KvWriteActor Shutting down: 0 queued messages to process [2018-10-11 03:54:36,54] [info] ServiceRegistryActor stopped [2018-10-11 03:54:36,54] [info] Database closed [2018-10-11 03:54:36,54] [info] Stream materializer shut down Workflow f99b0390-cc7a-43a0-a7fa-294cf7d512af transitioned to state Failed [2018-10-11 03:54:36,57] [info] Automatic shutdown of the async connection [2018-10-11 03:54:36,57] [info] Gracefully shutdown sentry threads. [2018-10-11 03:54:36,57] [info] Shutdown finished.
And here's the path after I activated conda environment: /home/groups/ashbym/mandy/anaconda3/envs/encode-chip-seq-pipeline/bin:/home/users/mkmwong/perl5/bin:/share/software/user/open/perl/5.26.0/bin:/home/groups/ashbym/mandy/anaconda3/bin:/share/software/user/srcc/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/dell/srvadmin/bin:/opt/dell/srvadmin/iSM/bin:/home/groups/ashbym/mandy/.bds:/home/groups/ashbym/mandy/anaconda3/bin/samtools:/home/groups/ashbym/mandy/bowtie2-2.3.4.3-linux-x86_64:/home/groups/ashbym/mandy/bedtools2/bin:/home/groups/ashbym/mandy/sratoolkit.2.9.2-centos_linux64/bin:/home/users/mkmwong/bin:/home/users/mkmwong/localperl/bin
Thank you !