leepc12
commented
3 years ago This looks like Conda and bowtie2 conflicts. I will release new caper + pipeline by end of this week. That will fix all problems.
Open asanghi7 opened 3 years ago
leepc12
commented
3 years ago This looks like Conda and bowtie2 conflicts. I will release new caper + pipeline by end of this week. That will fix all problems.
I tried the example json on the SCG cluster, and it exited with errors. I read in another issue that this type of error can occur if installed with mamba. However, I did not install with mamba. Please advise as soon as possible. We're hoping to submit this analysis very soon.
OS/Platform
2021-10-19 21:38:30,352 INFO - Running with database db.url = jdbc:hsqldb:mem:dcee96c5-7fbb-4cc4-b70b-102bbaf89454;shutdown=false;hsqldb.tx=mvcc 2021-10-19 21:38:43,506 INFO - Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000 2021-10-19 21:38:43,539 INFO - [RenameWorkflowOptionsInMetadata] 100% 2021-10-19 21:38:43,913 INFO - Running with database db.url = jdbc:hsqldb:mem:9beec382-9592-42ec-9fe5-c9442ad42f5f;shutdown=false;hsqldb.tx=mvcc 2021-10-19 21:38:44,792 INFO - Slf4jLogger started 2021-10-19 21:38:45,027 cromwell-system-akka.dispatchers.engine-dispatcher-8 INFO - Workflow heartbeat configuration: { "cromwellId" : "cromid-70c07ea", "heartbeatInterval" : "2 minutes", "ttl" : "10 minutes", "failureShutdownDuration" : "5 minutes", "writeBatchSize" : 10000, "writeThreshold" : 10000 } 2021-10-19 21:38:45,187 cromwell-system-akka.dispatchers.service-dispatcher-12 INFO - WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds. 2021-10-19 21:38:45,510 cromwell-system-akka.actor.default-dispatcher-6 INFO - KvWriteActor configured to flush with batch size 200 and process rate 5 seconds. 2021-10-19 21:38:45,576 WARN - 'docker.hash-lookup.gcr-api-queries-per-100-seconds' is being deprecated, use 'docker.hash-lookup.gcr.throttle' instead (see reference.conf) 2021-10-19 21:38:45,604 cromwell-system-akka.dispatchers.service-dispatcher-18 INFO - Metadata summary refreshing every 1 second. 2021-10-19 21:38:45,604 cromwell-system-akka.dispatchers.engine-dispatcher-36 INFO - CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds. 2021-10-19 21:38:45,756 cromwell-system-akka.dispatchers.engine-dispatcher-59 INFO - JobExecutionTokenDispenser - Distribution rate: 1 per 2 seconds. 2021-10-19 21:38:45,913 cromwell-system-akka.dispatchers.engine-dispatcher-8 INFO - SingleWorkflowRunnerActor: Version 59 2021-10-19 21:38:45,929 cromwell-system-akka.dispatchers.engine-dispatcher-8 INFO - SingleWorkflowRunnerActor: Submitting workflow 2021-10-19 21:38:46,033 cromwell-system-akka.dispatchers.api-dispatcher-60 INFO - Unspecified type (Unspecified version) workflow d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 submitted 2021-10-19 21:38:46,082 cromwell-system-akka.dispatchers.engine-dispatcher-48 INFO - SingleWorkflowRunnerActor: Workflow submitted UUID(d55a5c29-1c0c-4d31-82ab-47b89c1a37a5) 2021-10-19 21:38:46,091 cromwell-system-akka.dispatchers.engine-dispatcher-32 INFO - 1 new workflows fetched by cromid-70c07ea: d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 2021-10-19 21:38:46,097 cromwell-system-akka.dispatchers.engine-dispatcher-47 INFO - WorkflowManagerActor: Starting workflow UUID(d55a5c29-1c0c-4d31-82ab-47b89c1a37a5) 2021-10-19 21:38:46,111 cromwell-system-akka.dispatchers.engine-dispatcher-47 INFO - WorkflowManagerActor: Successfully started WorkflowActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 2021-10-19 21:38:46,111 cromwell-system-akka.dispatchers.engine-dispatcher-47 INFO - Retrieved 1 workflows from the WorkflowStoreActor 2021-10-19 21:38:46,142 cromwell-system-akka.dispatchers.engine-dispatcher-48 INFO - WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes. 2021-10-19 21:38:46,433 cromwell-system-akka.dispatchers.engine-dispatcher-48 INFO - MaterializeWorkflowDescriptorActor [UUID(d55a5c29)]: Parsing workflow as WDL 1.0 2021-10-19 21:38:51,236 cromwell-system-akka.dispatchers.engine-dispatcher-69 INFO - Not triggering log of token queue status. Effective log interval = None 2021-10-19 21:38:52,158 cromwell-system-akka.dispatchers.engine-dispatcher-48 INFO - MaterializeWorkflowDescriptorActor [UUID(d55a5c29)]: Call-to-Backend assignments: atac.pool_ta_pr1 -> slurm, atac.pool_ta -> slurm, atac.call_peak_ppr2 -> slurm, atac.bam2ta_no_dedup -> slurm, atac.macs2_signal_track -> slurm, atac.overlap -> slurm, atac.align_mito -> slurm, atac.frac_mito -> slurm, atac.idr_pr -> slurm, atac.call_peak_pooled -> slurm, atac.align -> slurm, atac.overlap_ppr -> slurm, atac.filter -> slurm, atac.fraglen_stat_pe -> slurm, atac.jsd -> slurm, atac.idr -> slurm, atac.tss_enrich -> slurm, atac.pool_ta_pr2 -> slurm, atac.annot_enrich -> slurm, atac.macs2_signal_track_pooled -> slurm, atac.overlap_pr -> slurm, atac.qc_report -> slurm, atac.compare_signal_to_roadmap -> slurm, atac.bam2ta -> slurm, atac.call_peak_pr2 -> slurm, atac.count_signal_track -> slurm, atac.read_genome_tsv -> slurm, atac.error_input_data -> slurm, atac.idr_ppr -> slurm, atac.spr -> slurm, atac.reproducibility_idr -> slurm, atac.count_signal_track_pooled -> slurm, atac.gc_bias -> slurm, atac.filter_no_dedup -> slurm, atac.xcor -> slurm, atac.call_peak_ppr1 -> slurm, atac.reproducibility_overlap -> slurm, atac.preseq -> slurm, atac.pool_blacklist -> slurm, atac.call_peak -> slurm, atac.call_peak_pr1 -> slurm 2021-10-19 21:38:52,419 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,422 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,425 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,426 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,427 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,433 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,434 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,435 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,435 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,436 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,437 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,438 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,441 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,442 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,442 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,443 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,446 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,447 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,447 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,448 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,448 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,449 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,450 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,450 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,450 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,451 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,451 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,451 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,452 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,452 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,452 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,453 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,454 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,454 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,455 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,455 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,456 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,456 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,457 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,457 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:52,458 cromwell-system-akka.dispatchers.backend-dispatcher-104 WARN - slurm [UUID(d55a5c29)]: Key/s [preemptible, disks] is/are not supported by backend. Unsupported attributes will not be part of job executions. 2021-10-19 21:38:56,910 cromwell-system-akka.dispatchers.engine-dispatcher-68 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Starting atac.read_genome_tsv 2021-10-19 21:38:57,795 cromwell-system-akka.dispatchers.engine-dispatcher-68 INFO - Assigned new job execution tokens to the following groups: d55a5c29: 1 2021-10-19 21:38:57,934 cromwell-system-akka.dispatchers.engine-dispatcher-32 INFO - d55a5c29-1c0c-4d31-82ab-47b89c1a37a5-EngineJobExecutionActor-atac.read_genome_tsv:NA:1 [UUID(d55a5c29)]: Could not copy a suitable cache hit for d55a5c29:atac.read_genome_tsv:-1:1. No copy attempts were made. 2021-10-19 21:38:57,970 cromwell-system-akka.dispatchers.backend-dispatcher-108 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.read_genome_tsv:NA:1]: Unrecognized runtime attribute keys: disks 2021-10-19 21:38:58,098 cromwell-system-akka.dispatchers.backend-dispatcher-108 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.read_genome_tsv:NA:1]: `echo "$(basename /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-read_genome_tsv/inputs/1395495084/hg38.local.tsv)" > genome_name
create empty files for all entries
touch ref_fa bowtie2_idx_tar chrsz gensz blacklist blacklist2 touch ref_mito_fa touch bowtie2_mito_idx_tar touch tss tss_enrich # for backward compatibility touch dnase prom enh reg2map reg2map_bed roadmap_meta touch mito_chr_name touch regex_bfilt_peak_chr_name
python <<CODE import os with open('/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-read_genome_tsv/inputs/1395495084/hg38.local.tsv','r') as fp: for line in fp: arr = line.strip('\n').split('\t') if arr: key, val = arr with open(key,'w') as fp2: fp2.write(val) CODE` 2021-10-19 21:38:58,558 cromwell-system-akka.dispatchers.backend-dispatcher-108 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.read_genome_tsv:NA:1]: executing: if [ -z \"$SINGULARITY_BINDPATH\" ]; then export SINGULARITY_BINDPATH=; fi; \ if [ -z \"$SINGULARITY_CACHEDIR\" ]; then export SINGULARITY_CACHEDIR=; fi;
ITER=0 until [ $ITER -ge 3 ]; do sbatch \ --export=ALL \ -J cromwell_d55a5c29_read_genome_tsv \ -D /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-read_genome_tsv \ -o /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-read_genome_tsv/execution/stdout \ -e /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-read_genome_tsv/execution/stderr \ -t 60 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=1 \ --mem=2048 \ \ --account mpsnyder \ \ \ --wrap "/bin/bash /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-read_genome_tsv/execution/script" \ && break ITER=$[$ITER+1] sleep 30 done 2021-10-19 21:39:00,557 cromwell-system-akka.dispatchers.backend-dispatcher-108 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.read_genome_tsv:NA:1]: job id: 6340 2021-10-19 21:39:00,562 cromwell-system-akka.dispatchers.backend-dispatcher-109 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.read_genome_tsv:NA:1]: Status change from - to WaitingForReturnCode 2021-10-19 21:39:11,287 cromwell-system-akka.dispatchers.backend-dispatcher-111 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.read_genome_tsv:NA:1]: Status change from WaitingForReturnCode to Done 2021-10-19 21:39:16,449 cromwell-system-akka.dispatchers.engine-dispatcher-55 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Starting atac.align (2 shards) 2021-10-19 21:39:17,777 cromwell-system-akka.dispatchers.engine-dispatcher-77 INFO - Assigned new job execution tokens to the following groups: d55a5c29: 1 2021-10-19 21:39:17,811 cromwell-system-akka.dispatchers.engine-dispatcher-74 INFO - d55a5c29-1c0c-4d31-82ab-47b89c1a37a5-EngineJobExecutionActor-atac.align:0:1 [UUID(d55a5c29)]: Could not copy a suitable cache hit for d55a5c29:atac.align:0:1. No copy attempts were made. 2021-10-19 21:39:17,848 cromwell-system-akka.dispatchers.backend-dispatcher-122 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:1]: Unrecognized runtime attribute keys: preemptible, disks 2021-10-19 21:39:17,939 cromwell-system-akka.dispatchers.backend-dispatcher-122 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:1]: `set -e
check if pipeline dependencies can be found
if [[ -z "$(which encode_task_trim_adapter.py 2> /dev/null || true)" ]] then echo -e "\n* Error: pipeline dependencies not found." 1>&2 echo 'Conda users: Did you activate Conda environment (conda activate encode-atac-seq-pipeline)?' 1>&2 echo ' Or did you install Conda and environment correctly (bash scripts/install_conda_env.sh)?' 1>&2 echo 'GCP/AWS/Docker users: Did you add --docker flag to Caper command line arg?' 1>&2 echo 'Singularity users: Did you add --singularity flag to Caper command line arg?' 1>&2 echo -e "\n" 1>&2 exit 3 fi
trim adapter
python3 $(which encode_task_trim_adapter.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/execution/write_tsv_a3574bbb5b4477d98ce577e7008e3164.tmp \ \ --adapters /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/execution/write_tsv_d41d8cd98f00b204e9800998ecf8427e.tmp \ --paired-end \ --auto-detect-adapter \ --cutadapt-param ' -e 0.1 -m 5' \ --nth 6
align on trimmed/merged fastqs
if [ 'bowtie2' == 'bowtie2' ]; then python3 $(which encode_task_bowtie2.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/inputs/1211538853/ENCFF110MCL.tar.gz \ R1/.fastq.gz \ R2/.fastq.gz \ --paired-end \ --multimapping 4 \ --mem-gb 6.796997847855092 \ --nth 6 fi
python3 $(which encode_task_post_align.py) \ R1/.fastq.gz $(ls .bam) \ --mito-chr-name chrM \ --chrsz /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/inputs/-258180794/GRCh38_EBV.chrom.sizes.tsv \ --mem-gb 6.796997847855092 \ --nth 6 rm -rf R1 R2` 2021-10-19 21:39:17,966 cromwell-system-akka.dispatchers.backend-dispatcher-122 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:1]: executing: if [ -z \"$SINGULARITY_BINDPATH\" ]; then export SINGULARITY_BINDPATH=; fi; \ if [ -z \"$SINGULARITY_CACHEDIR\" ]; then export SINGULARITY_CACHEDIR=; fi;
ITER=0 until [ $ITER -ge 3 ]; do sbatch \ --export=ALL \ -J cromwell_d55a5c29_align \ -D /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0 \ -o /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/execution/stdout \ -e /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=6 \ --mem=8700 \ \ --account mpsnyder \ \ \ --wrap "/bin/bash /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/execution/script" \ && break ITER=$[$ITER+1] sleep 30 done 2021-10-19 21:39:19,558 cromwell-system-akka.dispatchers.engine-dispatcher-74 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Starting atac.align_mito (2 shards) 2021-10-19 21:39:19,776 cromwell-system-akka.dispatchers.engine-dispatcher-81 INFO - Assigned new job execution tokens to the following groups: d55a5c29: 1 2021-10-19 21:39:19,790 cromwell-system-akka.dispatchers.engine-dispatcher-55 INFO - d55a5c29-1c0c-4d31-82ab-47b89c1a37a5-EngineJobExecutionActor-atac.align:1:1 [UUID(d55a5c29)]: Could not copy a suitable cache hit for d55a5c29:atac.align:1:1. No copy attempts were made. 2021-10-19 21:39:19,790 cromwell-system-akka.dispatchers.backend-dispatcher-121 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:1]: Unrecognized runtime attribute keys: preemptible, disks 2021-10-19 21:39:19,951 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:1]: `set -e
check if pipeline dependencies can be found
if [[ -z "$(which encode_task_trim_adapter.py 2> /dev/null || true)" ]] then echo -e "\n* Error: pipeline dependencies not found." 1>&2 echo 'Conda users: Did you activate Conda environment (conda activate encode-atac-seq-pipeline)?' 1>&2 echo ' Or did you install Conda and environment correctly (bash scripts/install_conda_env.sh)?' 1>&2 echo 'GCP/AWS/Docker users: Did you add --docker flag to Caper command line arg?' 1>&2 echo 'Singularity users: Did you add --singularity flag to Caper command line arg?' 1>&2 echo -e "\n" 1>&2 exit 3 fi
trim adapter
python3 $(which encode_task_trim_adapter.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/execution/write_tsv_08731ea344ef795342a3da331f5bdd54.tmp \ \ --adapters /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/execution/write_tsv_d41d8cd98f00b204e9800998ecf8427e.tmp \ --paired-end \ --auto-detect-adapter \ --cutadapt-param ' -e 0.1 -m 5' \ --nth 6
align on trimmed/merged fastqs
if [ 'bowtie2' == 'bowtie2' ]; then python3 $(which encode_task_bowtie2.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/inputs/1211538853/ENCFF110MCL.tar.gz \ R1/.fastq.gz \ R2/.fastq.gz \ --paired-end \ --multimapping 4 \ --mem-gb 6.797781316973269 \ --nth 6 fi
python3 $(which encode_task_post_align.py) \ R1/.fastq.gz $(ls .bam) \ --mito-chr-name chrM \ --chrsz /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/inputs/-258180794/GRCh38_EBV.chrom.sizes.tsv \ --mem-gb 6.797781316973269 \ --nth 6 rm -rf R1 R2` 2021-10-19 21:39:19,977 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:1]: executing: if [ -z \"$SINGULARITY_BINDPATH\" ]; then export SINGULARITY_BINDPATH=; fi; \ if [ -z \"$SINGULARITY_CACHEDIR\" ]; then export SINGULARITY_CACHEDIR=; fi;
ITER=0 until [ $ITER -ge 3 ]; do sbatch \ --export=ALL \ -J cromwell_d55a5c29_align \ -D /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1 \ -o /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/execution/stdout \ -e /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=6 \ --mem=8701 \ \ --account mpsnyder \ \ \ --wrap "/bin/bash /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/execution/script" \ && break ITER=$[$ITER+1] sleep 30 done 2021-10-19 21:39:20,535 cromwell-system-akka.dispatchers.backend-dispatcher-124 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:1]: job id: 6375 2021-10-19 21:39:20,536 cromwell-system-akka.dispatchers.backend-dispatcher-122 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:1]: job id: 6381 2021-10-19 21:39:20,563 cromwell-system-akka.dispatchers.backend-dispatcher-124 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:1]: Status change from - to WaitingForReturnCode 2021-10-19 21:39:20,564 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:1]: Status change from - to WaitingForReturnCode 2021-10-19 21:39:21,786 cromwell-system-akka.dispatchers.engine-dispatcher-54 INFO - Assigned new job execution tokens to the following groups: d55a5c29: 1 2021-10-19 21:39:21,798 cromwell-system-akka.dispatchers.engine-dispatcher-33 INFO - d55a5c29-1c0c-4d31-82ab-47b89c1a37a5-EngineJobExecutionActor-atac.align_mito:0:1 [UUID(d55a5c29)]: Could not copy a suitable cache hit for d55a5c29:atac.align_mito:0:1. No copy attempts were made. 2021-10-19 21:39:21,801 cromwell-system-akka.dispatchers.backend-dispatcher-124 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:1]: Unrecognized runtime attribute keys: preemptible, disks 2021-10-19 21:39:21,932 cromwell-system-akka.dispatchers.backend-dispatcher-124 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:1]: `set -e
check if pipeline dependencies can be found
if [[ -z "$(which encode_task_trim_adapter.py 2> /dev/null || true)" ]] then echo -e "\n* Error: pipeline dependencies not found." 1>&2 echo 'Conda users: Did you activate Conda environment (conda activate encode-atac-seq-pipeline)?' 1>&2 echo ' Or did you install Conda and environment correctly (bash scripts/install_conda_env.sh)?' 1>&2 echo 'GCP/AWS/Docker users: Did you add --docker flag to Caper command line arg?' 1>&2 echo 'Singularity users: Did you add --singularity flag to Caper command line arg?' 1>&2 echo -e "\n" 1>&2 exit 3 fi
trim adapter
python3 $(which encode_task_trim_adapter.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/execution/write_tsv_af71ef434d20472344a70357880457d9.tmp \ \ --adapters /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/execution/write_tsv_d41d8cd98f00b204e9800998ecf8427e.tmp \ --paired-end \ --auto-detect-adapter \ --cutadapt-param ' -e 0.1 -m 5' \ --nth 6
align on trimmed/merged fastqs
if [ 'bowtie2' == 'bowtie2' ]; then python3 $(which encode_task_bowtie2.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/inputs/1029504244/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz \ R1/.fastq.gz \ R2/.fastq.gz \ --paired-end \ --multimapping 4 \ --mem-gb 4.00367325887084 \ --nth 6 fi
python3 $(which encode_task_post_align.py) \ R1/.fastq.gz $(ls .bam) \ --mito-chr-name chrM \ --chrsz /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/inputs/-258180794/GRCh38_EBV.chrom.sizes.tsv \ --mem-gb 4.00367325887084 \ --nth 6 rm -rf R1 R2` 2021-10-19 21:39:21,980 cromwell-system-akka.dispatchers.backend-dispatcher-124 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:1]: executing: if [ -z \"$SINGULARITY_BINDPATH\" ]; then export SINGULARITY_BINDPATH=; fi; \ if [ -z \"$SINGULARITY_CACHEDIR\" ]; then export SINGULARITY_CACHEDIR=; fi;
ITER=0 until [ $ITER -ge 3 ]; do sbatch \ --export=ALL \ -J cromwell_d55a5c29_align_mito \ -D /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0 \ -o /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/execution/stdout \ -e /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=6 \ --mem=5124 \ \ --account mpsnyder \ \ \ --wrap "/bin/bash /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/execution/script" \ && break ITER=$[$ITER+1] sleep 30 done 2021-10-19 21:39:23,776 cromwell-system-akka.dispatchers.engine-dispatcher-66 INFO - Assigned new job execution tokens to the following groups: d55a5c29: 1 2021-10-19 21:39:23,801 cromwell-system-akka.dispatchers.engine-dispatcher-8 INFO - d55a5c29-1c0c-4d31-82ab-47b89c1a37a5-EngineJobExecutionActor-atac.align_mito:1:1 [UUID(d55a5c29)]: Could not copy a suitable cache hit for d55a5c29:atac.align_mito:1:1. No copy attempts were made. 2021-10-19 21:39:23,802 cromwell-system-akka.dispatchers.backend-dispatcher-124 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:1]: Unrecognized runtime attribute keys: preemptible, disks 2021-10-19 21:39:23,964 cromwell-system-akka.dispatchers.backend-dispatcher-124 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:1]: `set -e
check if pipeline dependencies can be found
if [[ -z "$(which encode_task_trim_adapter.py 2> /dev/null || true)" ]] then echo -e "\n* Error: pipeline dependencies not found." 1>&2 echo 'Conda users: Did you activate Conda environment (conda activate encode-atac-seq-pipeline)?' 1>&2 echo ' Or did you install Conda and environment correctly (bash scripts/install_conda_env.sh)?' 1>&2 echo 'GCP/AWS/Docker users: Did you add --docker flag to Caper command line arg?' 1>&2 echo 'Singularity users: Did you add --singularity flag to Caper command line arg?' 1>&2 echo -e "\n" 1>&2 exit 3 fi
trim adapter
python3 $(which encode_task_trim_adapter.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/execution/write_tsv_92ef5aa7d4bf099b1a405e5db8b61eb2.tmp \ \ --adapters /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/execution/write_tsv_d41d8cd98f00b204e9800998ecf8427e.tmp \ --paired-end \ --auto-detect-adapter \ --cutadapt-param ' -e 0.1 -m 5' \ --nth 6
align on trimmed/merged fastqs
if [ 'bowtie2' == 'bowtie2' ]; then python3 $(which encode_task_bowtie2.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/inputs/1029504244/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz \ R1/.fastq.gz \ R2/.fastq.gz \ --paired-end \ --multimapping 4 \ --mem-gb 4.004456727989019 \ --nth 6 fi
python3 $(which encode_task_post_align.py) \ R1/.fastq.gz $(ls .bam) \ --mito-chr-name chrM \ --chrsz /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/inputs/-258180794/GRCh38_EBV.chrom.sizes.tsv \ --mem-gb 4.004456727989019 \ --nth 6 rm -rf R1 R2` 2021-10-19 21:39:23,974 cromwell-system-akka.dispatchers.backend-dispatcher-124 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:1]: executing: if [ -z \"$SINGULARITY_BINDPATH\" ]; then export SINGULARITY_BINDPATH=; fi; \ if [ -z \"$SINGULARITY_CACHEDIR\" ]; then export SINGULARITY_CACHEDIR=; fi;
ITER=0 until [ $ITER -ge 3 ]; do sbatch \ --export=ALL \ -J cromwell_d55a5c29_align_mito \ -D /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1 \ -o /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/execution/stdout \ -e /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=6 \ --mem=5125 \ \ --account mpsnyder \ \ \ --wrap "/bin/bash /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/execution/script" \ && break ITER=$[$ITER+1] sleep 30 done 2021-10-19 21:39:25,526 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:1]: job id: 6392 2021-10-19 21:39:25,528 cromwell-system-akka.dispatchers.backend-dispatcher-126 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:1]: job id: 6399 2021-10-19 21:39:25,528 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:1]: Status change from - to WaitingForReturnCode 2021-10-19 21:39:25,538 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:1]: Status change from - to WaitingForReturnCode 2021-10-19 21:40:01,464 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:1]: Status change from WaitingForReturnCode to Done 2021-10-19 21:40:05,525 cromwell-system-akka.dispatchers.engine-dispatcher-10 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Retrying job execution for atac.align_mito:0:2 2021-10-19 21:40:06,288 cromwell-system-akka.dispatchers.engine-dispatcher-72 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Starting atac.align_mito 2021-10-19 21:40:07,763 cromwell-system-akka.dispatchers.engine-dispatcher-72 INFO - Assigned new job execution tokens to the following groups: d55a5c29: 1 2021-10-19 21:40:07,792 cromwell-system-akka.dispatchers.backend-dispatcher-127 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:2]: Unrecognized runtime attribute keys: preemptible, disks 2021-10-19 21:40:07,878 cromwell-system-akka.dispatchers.backend-dispatcher-127 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:2]: `set -e
check if pipeline dependencies can be found
if [[ -z "$(which encode_task_trim_adapter.py 2> /dev/null || true)" ]] then echo -e "\n* Error: pipeline dependencies not found." 1>&2 echo 'Conda users: Did you activate Conda environment (conda activate encode-atac-seq-pipeline)?' 1>&2 echo ' Or did you install Conda and environment correctly (bash scripts/install_conda_env.sh)?' 1>&2 echo 'GCP/AWS/Docker users: Did you add --docker flag to Caper command line arg?' 1>&2 echo 'Singularity users: Did you add --singularity flag to Caper command line arg?' 1>&2 echo -e "\n" 1>&2 exit 3 fi
trim adapter
python3 $(which encode_task_trim_adapter.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/write_tsv_c9eb94be28453316652cbd92605fbb87.tmp \ \ --adapters /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/write_tsv_d41d8cd98f00b204e9800998ecf8427e.tmp \ --paired-end \ --auto-detect-adapter \ --cutadapt-param ' -e 0.1 -m 5' \ --nth 6
align on trimmed/merged fastqs
if [ 'bowtie2' == 'bowtie2' ]; then python3 $(which encode_task_bowtie2.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/inputs/1029504244/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz \ R1/.fastq.gz \ R2/.fastq.gz \ --paired-end \ --multimapping 4 \ --mem-gb 4.00367325887084 \ --nth 6 fi
python3 $(which encode_task_post_align.py) \ R1/.fastq.gz $(ls .bam) \ --mito-chr-name chrM \ --chrsz /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/inputs/-258180794/GRCh38_EBV.chrom.sizes.tsv \ --mem-gb 4.00367325887084 \ --nth 6 rm -rf R1 R2` 2021-10-19 21:40:07,917 cromwell-system-akka.dispatchers.backend-dispatcher-127 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:2]: executing: if [ -z \"$SINGULARITY_BINDPATH\" ]; then export SINGULARITY_BINDPATH=; fi; \ if [ -z \"$SINGULARITY_CACHEDIR\" ]; then export SINGULARITY_CACHEDIR=; fi;
ITER=0 until [ $ITER -ge 3 ]; do sbatch \ --export=ALL \ -J cromwell_d55a5c29_align_mito \ -D /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2 \ -o /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/stdout \ -e /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=6 \ --mem=5124 \ \ --account mpsnyder \ \ \ --wrap "/bin/bash /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/script" \ && break ITER=$[$ITER+1] sleep 30 done 2021-10-19 21:40:10,525 cromwell-system-akka.dispatchers.backend-dispatcher-122 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:2]: job id: 6584 2021-10-19 21:40:10,526 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:2]: Status change from - to WaitingForReturnCode 2021-10-19 21:40:10,774 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:1]: Status change from WaitingForReturnCode to Done 2021-10-19 21:40:15,528 cromwell-system-akka.dispatchers.engine-dispatcher-75 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Retrying job execution for atac.align_mito:1:2 2021-10-19 21:40:16,446 cromwell-system-akka.dispatchers.engine-dispatcher-77 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Starting atac.align_mito 2021-10-19 21:40:17,765 cromwell-system-akka.dispatchers.engine-dispatcher-58 INFO - Assigned new job execution tokens to the following groups: d55a5c29: 1 2021-10-19 21:40:17,824 cromwell-system-akka.dispatchers.backend-dispatcher-126 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:2]: Unrecognized runtime attribute keys: preemptible, disks 2021-10-19 21:40:17,960 cromwell-system-akka.dispatchers.backend-dispatcher-126 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:2]: `set -e
check if pipeline dependencies can be found
if [[ -z "$(which encode_task_trim_adapter.py 2> /dev/null || true)" ]] then echo -e "\n* Error: pipeline dependencies not found." 1>&2 echo 'Conda users: Did you activate Conda environment (conda activate encode-atac-seq-pipeline)?' 1>&2 echo ' Or did you install Conda and environment correctly (bash scripts/install_conda_env.sh)?' 1>&2 echo 'GCP/AWS/Docker users: Did you add --docker flag to Caper command line arg?' 1>&2 echo 'Singularity users: Did you add --singularity flag to Caper command line arg?' 1>&2 echo -e "\n" 1>&2 exit 3 fi
trim adapter
python3 $(which encode_task_trim_adapter.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/write_tsv_11aec7d624710d0c4fe789ef1822853b.tmp \ \ --adapters /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/write_tsv_d41d8cd98f00b204e9800998ecf8427e.tmp \ --paired-end \ --auto-detect-adapter \ --cutadapt-param ' -e 0.1 -m 5' \ --nth 6
align on trimmed/merged fastqs
if [ 'bowtie2' == 'bowtie2' ]; then python3 $(which encode_task_bowtie2.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/inputs/1029504244/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz \ R1/.fastq.gz \ R2/.fastq.gz \ --paired-end \ --multimapping 4 \ --mem-gb 4.004456727989019 \ --nth 6 fi
python3 $(which encode_task_post_align.py) \ R1/.fastq.gz $(ls .bam) \ --mito-chr-name chrM \ --chrsz /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/inputs/-258180794/GRCh38_EBV.chrom.sizes.tsv \ --mem-gb 4.004456727989019 \ --nth 6 rm -rf R1 R2` 2021-10-19 21:40:17,982 cromwell-system-akka.dispatchers.backend-dispatcher-126 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:2]: executing: if [ -z \"$SINGULARITY_BINDPATH\" ]; then export SINGULARITY_BINDPATH=; fi; \ if [ -z \"$SINGULARITY_CACHEDIR\" ]; then export SINGULARITY_CACHEDIR=; fi;
ITER=0 until [ $ITER -ge 3 ]; do sbatch \ --export=ALL \ -J cromwell_d55a5c29_align_mito \ -D /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2 \ -o /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/stdout \ -e /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=6 \ --mem=5125 \ \ --account mpsnyder \ \ \ --wrap "/bin/bash /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/script" \ && break ITER=$[$ITER+1] sleep 30 done 2021-10-19 21:40:20,517 cromwell-system-akka.dispatchers.backend-dispatcher-126 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:2]: job id: 6599 2021-10-19 21:40:20,519 cromwell-system-akka.dispatchers.backend-dispatcher-128 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:2]: Status change from - to WaitingForReturnCode 2021-10-19 21:40:27,845 cromwell-system-akka.dispatchers.backend-dispatcher-126 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:1]: Status change from WaitingForReturnCode to Done 2021-10-19 21:40:30,517 cromwell-system-akka.dispatchers.engine-dispatcher-77 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Retrying job execution for atac.align:0:2 2021-10-19 21:40:30,646 cromwell-system-akka.dispatchers.engine-dispatcher-81 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Starting atac.align 2021-10-19 21:40:31,763 cromwell-system-akka.dispatchers.engine-dispatcher-71 INFO - Assigned new job execution tokens to the following groups: d55a5c29: 1 2021-10-19 21:40:31,786 cromwell-system-akka.dispatchers.backend-dispatcher-129 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:2]: Unrecognized runtime attribute keys: preemptible, disks 2021-10-19 21:40:31,851 cromwell-system-akka.dispatchers.backend-dispatcher-129 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:2]: `set -e
check if pipeline dependencies can be found
if [[ -z "$(which encode_task_trim_adapter.py 2> /dev/null || true)" ]] then echo -e "\n* Error: pipeline dependencies not found." 1>&2 echo 'Conda users: Did you activate Conda environment (conda activate encode-atac-seq-pipeline)?' 1>&2 echo ' Or did you install Conda and environment correctly (bash scripts/install_conda_env.sh)?' 1>&2 echo 'GCP/AWS/Docker users: Did you add --docker flag to Caper command line arg?' 1>&2 echo 'Singularity users: Did you add --singularity flag to Caper command line arg?' 1>&2 echo -e "\n" 1>&2 exit 3 fi
trim adapter
python3 $(which encode_task_trim_adapter.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/write_tsv_34b395693e2f7c0ae3a5cd074837b445.tmp \ \ --adapters /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/write_tsv_d41d8cd98f00b204e9800998ecf8427e.tmp \ --paired-end \ --auto-detect-adapter \ --cutadapt-param ' -e 0.1 -m 5' \ --nth 6
align on trimmed/merged fastqs
if [ 'bowtie2' == 'bowtie2' ]; then python3 $(which encode_task_bowtie2.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/inputs/1211538853/ENCFF110MCL.tar.gz \ R1/.fastq.gz \ R2/.fastq.gz \ --paired-end \ --multimapping 4 \ --mem-gb 6.796997847855092 \ --nth 6 fi
python3 $(which encode_task_post_align.py) \ R1/.fastq.gz $(ls .bam) \ --mito-chr-name chrM \ --chrsz /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/inputs/-258180794/GRCh38_EBV.chrom.sizes.tsv \ --mem-gb 6.796997847855092 \ --nth 6 rm -rf R1 R2` 2021-10-19 21:40:31,890 cromwell-system-akka.dispatchers.backend-dispatcher-129 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:2]: executing: if [ -z \"$SINGULARITY_BINDPATH\" ]; then export SINGULARITY_BINDPATH=; fi; \ if [ -z \"$SINGULARITY_CACHEDIR\" ]; then export SINGULARITY_CACHEDIR=; fi;
ITER=0 until [ $ITER -ge 3 ]; do sbatch \ --export=ALL \ -J cromwell_d55a5c29_align \ -D /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2 \ -o /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/stdout \ -e /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=6 \ --mem=8700 \ \ --account mpsnyder \ \ \ --wrap "/bin/bash /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/script" \ && break ITER=$[$ITER+1] sleep 30 done 2021-10-19 21:40:35,536 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:2]: job id: 6743 2021-10-19 21:40:35,538 cromwell-system-akka.dispatchers.backend-dispatcher-126 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:2]: Status change from - to WaitingForReturnCode 2021-10-19 21:40:38,745 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:1]: Status change from WaitingForReturnCode to Done 2021-10-19 21:40:40,527 cromwell-system-akka.dispatchers.engine-dispatcher-77 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Retrying job execution for atac.align:1:2 2021-10-19 21:40:40,838 cromwell-system-akka.dispatchers.engine-dispatcher-9 INFO - WorkflowExecutionActor-d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 [UUID(d55a5c29)]: Starting atac.align 2021-10-19 21:40:41,776 cromwell-system-akka.dispatchers.engine-dispatcher-81 INFO - Assigned new job execution tokens to the following groups: d55a5c29: 1 2021-10-19 21:40:41,804 cromwell-system-akka.dispatchers.backend-dispatcher-121 WARN - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:2]: Unrecognized runtime attribute keys: preemptible, disks 2021-10-19 21:40:41,935 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:2]: `set -e
check if pipeline dependencies can be found
if [[ -z "$(which encode_task_trim_adapter.py 2> /dev/null || true)" ]] then echo -e "\n* Error: pipeline dependencies not found." 1>&2 echo 'Conda users: Did you activate Conda environment (conda activate encode-atac-seq-pipeline)?' 1>&2 echo ' Or did you install Conda and environment correctly (bash scripts/install_conda_env.sh)?' 1>&2 echo 'GCP/AWS/Docker users: Did you add --docker flag to Caper command line arg?' 1>&2 echo 'Singularity users: Did you add --singularity flag to Caper command line arg?' 1>&2 echo -e "\n" 1>&2 exit 3 fi
trim adapter
python3 $(which encode_task_trim_adapter.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/write_tsv_fe212f8af0ea52db840a81aa9f99377d.tmp \ \ --adapters /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/write_tsv_d41d8cd98f00b204e9800998ecf8427e.tmp \ --paired-end \ --auto-detect-adapter \ --cutadapt-param ' -e 0.1 -m 5' \ --nth 6
align on trimmed/merged fastqs
if [ 'bowtie2' == 'bowtie2' ]; then python3 $(which encode_task_bowtie2.py) \ /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/inputs/1211538853/ENCFF110MCL.tar.gz \ R1/.fastq.gz \ R2/.fastq.gz \ --paired-end \ --multimapping 4 \ --mem-gb 6.797781316973269 \ --nth 6 fi
python3 $(which encode_task_post_align.py) \ R1/.fastq.gz $(ls .bam) \ --mito-chr-name chrM \ --chrsz /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/inputs/-258180794/GRCh38_EBV.chrom.sizes.tsv \ --mem-gb 6.797781316973269 \ --nth 6 rm -rf R1 R2` 2021-10-19 21:40:41,970 cromwell-system-akka.dispatchers.backend-dispatcher-121 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:2]: executing: if [ -z \"$SINGULARITY_BINDPATH\" ]; then export SINGULARITY_BINDPATH=; fi; \ if [ -z \"$SINGULARITY_CACHEDIR\" ]; then export SINGULARITY_CACHEDIR=; fi;
ITER=0 until [ $ITER -ge 3 ]; do sbatch \ --export=ALL \ -J cromwell_d55a5c29_align \ -D /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2 \ -o /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/stdout \ -e /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/stderr \ -t 2880 \ -n 1 \ --ntasks-per-node=1 \ --cpus-per-task=6 \ --mem=8701 \ \ --account mpsnyder \ \ \ --wrap "/bin/bash /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/script" \ && break ITER=$[$ITER+1] sleep 30 done 2021-10-19 21:40:43,074 cromwell-system-akka.dispatchers.backend-dispatcher-122 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:0:2]: Status change from WaitingForReturnCode to Done 2021-10-19 21:40:45,515 cromwell-system-akka.dispatchers.backend-dispatcher-122 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:2]: job id: 6760 2021-10-19 21:40:45,517 cromwell-system-akka.dispatchers.backend-dispatcher-126 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:2]: Status change from - to WaitingForReturnCode 2021-10-19 21:41:03,834 cromwell-system-akka.dispatchers.backend-dispatcher-122 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align_mito:1:2]: Status change from WaitingForReturnCode to Done 2021-10-19 21:41:36,859 cromwell-system-akka.dispatchers.backend-dispatcher-122 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:0:2]: Status change from WaitingForReturnCode to Done 2021-10-19 21:42:07,558 cromwell-system-akka.dispatchers.backend-dispatcher-128 INFO - BackgroundConfigAsyncJobExecutionActor [UUID(d55a5c29)atac.align:1:2]: Status change from WaitingForReturnCode to Done 2021-10-19 21:42:08,060 cromwell-system-akka.dispatchers.engine-dispatcher-10 INFO - WorkflowManagerActor: Workflow d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 failed (during ExecutingWorkflowState): Job atac.align_mito:0:2 exited with return code 1 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details. Check the content of stderr for potential additional information: /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/stderr. [First 3000 bytes]:Traceback (most recent call last): File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 192, in
main()
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 169, in main
args.out_dir)
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 102, in bowtie2_pe
tmp_bam=tmp_bam,
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_lib_common.py", line 359, in run_shell_cmd
raise Exception(err_str)
Exception: PID=34629, PGID=34629, RC=127, DURATION_SEC=0.2
STDERR=/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/bowtie2-align-s: symbol lookup error: /home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/bowtie2-align-s: undefined symbol: _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv
(ERR): Description of arguments failed!
Exiting now ...
STDOUT=
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/.samstats.qc': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/.read_length.txt': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/.bai': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-0/attempt-2/execution/non_mito/.samstats.qc': No such file or directory
Job atac.align_mito:1:2 exited with return code 1 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details. Check the content of stderr for potential additional information: /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/stderr. [First 3000 bytes]:Traceback (most recent call last): File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 192, in
main()
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 169, in main
args.out_dir)
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 102, in bowtie2_pe
tmp_bam=tmp_bam,
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_lib_common.py", line 359, in run_shell_cmd
raise Exception(err_str)
Exception: PID=15947, PGID=15947, RC=127, DURATION_SEC=0.1
STDERR=/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/bowtie2-align-s: symbol lookup error: /home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/bowtie2-align-s: undefined symbol: _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv
(ERR): Description of arguments failed!
Exiting now ...
STDOUT=
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/.samstats.qc': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/.read_length.txt': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/.bai': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align_mito/shard-1/attempt-2/execution/non_mito/.samstats.qc': No such file or directory
Job atac.align:0:2 exited with return code 1 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details. Check the content of stderr for potential additional information: /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/stderr. [First 3000 bytes]:Traceback (most recent call last): File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 192, in
main()
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 169, in main
args.out_dir)
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 102, in bowtie2_pe
tmp_bam=tmp_bam,
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_lib_common.py", line 359, in run_shell_cmd
raise Exception(err_str)
Exception: PID=1425, PGID=1425, RC=127, DURATION_SEC=0.2
STDERR=/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/bowtie2-align-s: symbol lookup error: /home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/bowtie2-align-s: undefined symbol: _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv
(ERR): Description of arguments failed!
Exiting now ...
STDOUT=
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/.samstats.qc': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/.read_length.txt': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/.bai': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-0/attempt-2/execution/non_mito/.samstats.qc': No such file or directory
Job atac.align:1:2 exited with return code 1 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details. Check the content of stderr for potential additional information: /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/stderr. [First 3000 bytes]:Traceback (most recent call last): File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 192, in
main()
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 169, in main
args.out_dir)
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_task_bowtie2.py", line 102, in bowtie2_pe
tmp_bam=tmp_bam,
File "/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/encode_lib_common.py", line 359, in run_shell_cmd
raise Exception(err_str)
Exception: PID=33423, PGID=33423, RC=127, DURATION_SEC=0.3
STDERR=/home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/bowtie2-align-s: symbol lookup error: /home/asanghi/miniconda3/envs/encode-atac-seq-pipeline/bin/bowtie2-align-s: undefined symbol: _ZN3tbb10interface78internal15task_arena_base19internal_initializeEv
(ERR): Description of arguments failed!
Exiting now ...
STDOUT=
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/.samstats.qc': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/.read_length.txt': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/.bai': No such file or directory
ln: failed to access '/oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/atac/d55a5c29-1c0c-4d31-82ab-47b89c1a37a5/call-align/shard-1/attempt-2/execution/non_mito/.samstats.qc': No such file or directory
2021-10-19 21:42:10,231 cromwell-system-akka.dispatchers.engine-dispatcher-10 INFO - WorkflowManagerActor: Workflow actor for d55a5c29-1c0c-4d31-82ab-47b89c1a37a5 completed with status 'Failed'. The workflow will be removed from the workflow store. 2021-10-19 21:42:28,846 cromwell-system-akka.dispatchers.engine-dispatcher-10 INFO - SingleWorkflowRunnerActor workflow finished with status 'Failed'. 2021-10-19 21:42:30,777 cromwell-system-akka.dispatchers.engine-dispatcher-10 INFO - SingleWorkflowRunnerActor writing metadata to /oak/stanford/scg/lab_mpsnyder/asanghi/atac-seq-pipeline/example_input_json/test_output/.caper_tmp/atac/20211019_213421_394553/metadata.json 2021-10-19 21:42:30,958 INFO - Workflow polling stopped 2021-10-19 21:42:30,962 INFO - Shutting down WorkflowStoreActor - Timeout = 5 seconds 2021-10-19 21:42:30,962 INFO - 0 workflows released by cromid-70c07ea 2021-10-19 21:42:30,966 cromwell-system-akka.dispatchers.engine-dispatcher-65 INFO - Aborting all running workflows. 2021-10-19 21:42:30,969 INFO - Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds 2021-10-19 21:42:30,973 INFO - Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds 2021-10-19 21:42:30,973 INFO - WorkflowStoreActor stopped 2021-10-19 21:42:30,974 INFO - JobExecutionTokenDispenser stopped 2021-10-19 21:42:30,974 INFO - WorkflowLogCopyRouter stopped 2021-10-19 21:42:30,974 INFO - Shutting down WorkflowManagerActor - Timeout = 3600 seconds 2021-10-19 21:42:30,974 cromwell-system-akka.dispatchers.engine-dispatcher-10 INFO - WorkflowManagerActor: All workflows finished 2021-10-19 21:42:30,975 INFO - WorkflowManagerActor stopped 2021-10-19 21:42:31,804 INFO - Connection pools shut down 2021-10-19 21:42:31,804 INFO - Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds 2021-10-19 21:42:31,805 INFO - Shutting down JobStoreActor - Timeout = 1800 seconds 2021-10-19 21:42:31,805 INFO - Shutting down CallCacheWriteActor - Timeout = 1800 seconds 2021-10-19 21:42:31,806 INFO - CallCacheWriteActor Shutting down: 0 queued messages to process 2021-10-19 21:42:31,806 INFO - Shutting down ServiceRegistryActor - Timeout = 1800 seconds 2021-10-19 21:42:31,806 INFO - Shutting down DockerHashActor - Timeout = 1800 seconds 2021-10-19 21:42:31,809 INFO - Shutting down IoProxy - Timeout = 1800 seconds 2021-10-19 21:42:31,818 INFO - SubWorkflowStoreActor stopped 2021-10-19 21:42:31,828 INFO - JobStoreActor stopped 2021-10-19 21:42:31,829 INFO - CallCacheWriteActor stopped 2021-10-19 21:42:31,829 INFO - IoProxy stopped 2021-10-19 21:42:31,834 INFO - WriteMetadataActor Shutting down: 0 queued messages to process 2021-10-19 21:42:31,835 INFO - KvWriteActor Shutting down: 0 queued messages to process 2021-10-19 21:42:31,843 INFO - ServiceRegistryActor stopped 2021-10-19 21:42:31,951 INFO - DockerHashActor stopped 2021-10-19 21:42:32,037 INFO - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false 2021-10-19 21:42:32,038 INFO - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false 2021-10-19 21:42:32,038 INFO - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false 2021-10-19 21:42:32,038 INFO - Shutting down connection pool: curAllocated=0 idleQueues.size=0 waitQueue.size=0 maxWaitQueueLimit=256 closed=false 2021-10-19 21:42:32,058 INFO - Database closed 2021-10-19 21:42:32,059 INFO - Stream materializer shut down 2021-10-19 21:42:32,060 INFO - WDL HTTP import resolver closed