search

ENCODE-DCC / chip-seq-pipeline2

ENCODE ChIP-seq pipeline
MIT License
234 stars 123 forks source link

|ERROR| Failed to parse WDL with miniwdl, |ERROR| Conda environment name not found in WDL metadata #271

Closed shirleytemples closed 2 years ago

shirleytemples commented 2 years ago

Hi, when I ran the command caper run chip.wdl -i https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI_subsampled_chr19_only.json --conda

I get the error message|ERROR| Failed to parse WDL with miniwdl, Conda environment name not found in WDL metadata.

Any suggestions would be much appreciated!

I included the entire error message below for reference.

2022-03-05 00:44:55,190|caper.cli|INFO| Cromwell stdout: /home/shirleytemples/chip-seq-pipeline2/cromwell.out.59
2022-03-05 00:44:55,195|caper.caper_base|INFO| Creating a timestamped temporary directory. /home/shirleytemples/chip-seq-pipeline2/.caper_tmp/chip/20220305_004455_192720
2022-03-05 00:44:55,196|caper.caper_runner|INFO| Localizing files on work_dir. /home/shirleytemples/chip-seq-pipeline2/.caper_tmp/chip/20220305_004455_192720
2022-03-05 00:44:55,706|autouri.autouri|INFO| cp: (6dbd4d75) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38_chr19_chrM/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.fasta.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/dcc664bb3e886aa9127f3bd8522bff3c/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.fasta.gz
2022-03-05 00:44:56,167|autouri.autouri|INFO| cp: (6dbd4d75) skipped due to md5_match. md5=fbd798407e251364cf52712b461de7bd
2022-03-05 00:44:56,172|autouri.autouri|INFO| cp: (4353037b) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38_chr19_chrM/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.chrM.fa.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/be275eb60876cf28f4600fb8589afe42/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.chrM.fa.gz
2022-03-05 00:44:56,474|autouri.autouri|INFO| cp: (4353037b) skipped due to md5_match. md5=05297d96dd1f7cfb45a7b637d6dd7036
2022-03-05 00:44:56,481|autouri.autouri|INFO| cp: (01804529) started. src=https://www.encodeproject.org/files/ENCFF547MET/@@download/ENCFF547MET.bed.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/da42a1c4b0f9d3225600da56b2c9571d/ENCFF547MET.bed.gz
2022-03-05 00:44:57,110|autouri.autouri|INFO| cp: (01804529) skipped due to md5_match. md5=5d19eeb4197ae10540252c61399f3131
2022-03-05 00:44:57,116|autouri.autouri|INFO| cp: (a3339ce6) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38_chr19_chrM/hg38_chr19_chrM.chrom.sizes, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/f0e538218d74ce7a3850c9605caf2e23/hg38_chr19_chrM.chrom.sizes
2022-03-05 00:44:57,367|autouri.autouri|INFO| cp: (a3339ce6) skipped due to md5_match. md5=a068b95e53225dfb124590e10697d692
2022-03-05 00:44:57,377|autouri.autouri|INFO| cp: (6bd317b1) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38_chr19_chrM/bowtie2_index/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.fasta.tar, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/7b07a4b54cf5f715b174d8e9164d7d9c/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.fasta.tar
2022-03-05 00:44:57,858|autouri.autouri|INFO| cp: (6bd317b1) skipped due to md5_match. md5=22f0a7a6c0496b39f329f414fb192f75
2022-03-05 00:44:57,865|autouri.autouri|INFO| cp: (279ccd0a) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38_chr19_chrM/bowtie2_index/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.chrM.fa.tar, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/b44489aed009d7741fd44131dcc6b7c8/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.chrM.fa.tar
2022-03-05 00:44:58,196|autouri.autouri|INFO| cp: (279ccd0a) skipped due to md5_match. md5=c1b0155acebcd0f4d81ca79e8ec8bc79
2022-03-05 00:44:58,201|autouri.autouri|INFO| cp: (a5951eee) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38_chr19_chrM/bwa_index/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.fasta.tar, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/542354fad05fedb3cae8c97381031035/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.fasta.tar
2022-03-05 00:44:58,670|autouri.autouri|INFO| cp: (a5951eee) skipped due to md5_match. md5=d13bc228457d7b5347383658d1c2c4e8
2022-03-05 00:44:58,677|autouri.autouri|INFO| cp: (93f61117) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38_chr19_chrM/bwa_index/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.chrM.fa.tar, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/9b649793fb593506a76c3141180ea657/GRCh38_no_alt_analysis_set_GCA_000001405.15.chr19_chrM.chrM.fa.tar
2022-03-05 00:44:59,035|autouri.autouri|INFO| cp: (93f61117) skipped due to md5_match. md5=d7a5706a25bcd14874cac8e24b006e22
2022-03-05 00:44:59,041|autouri.autouri|INFO| cp: (68fc1758) started. src=https://www.encodeproject.org/files/ENCFF493CCB/@@download/ENCFF493CCB.bed.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/0fa7d04b32e66fa02fb2c1ae39e41447/ENCFF493CCB.bed.gz
2022-03-05 00:44:59,818|autouri.autouri|INFO| cp: (68fc1758) skipped due to md5_match. md5=aca8cf959206aa3ad257fc46dc783266
2022-03-05 00:44:59,824|autouri.autouri|INFO| cp: (2a5a74c5) started. src=https://www.encodeproject.org/files/ENCFF304XEX/@@download/ENCFF304XEX.bed.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/805e179275a9c0fb7a37def40c4312d1/ENCFF304XEX.bed.gz
2022-03-05 00:45:01,172|autouri.autouri|INFO| cp: (2a5a74c5) skipped due to name_size_match. size=14377496, mt=1592463730.0
2022-03-05 00:45:01,179|autouri.autouri|INFO| cp: (4c760c4d) started. src=https://www.encodeproject.org/files/ENCFF140XLU/@@download/ENCFF140XLU.bed.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/0cbd2c602ddad252bc39729fc8a29286/ENCFF140XLU.bed.gz
2022-03-05 00:45:01,960|autouri.autouri|INFO| cp: (4c760c4d) skipped due to md5_match. md5=91047588129069ff91ec1b0664179f8e
2022-03-05 00:45:01,966|autouri.autouri|INFO| cp: (aa339028) started. src=https://www.encodeproject.org/files/ENCFF212UAV/@@download/ENCFF212UAV.bed.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/1d3aa436b05f16a509edb94789c061d3/ENCFF212UAV.bed.gz
2022-03-05 00:45:03,095|autouri.autouri|INFO| cp: (aa339028) skipped due to name_size_match. size=18381891, mt=1592463727.0
2022-03-05 00:45:03,101|autouri.autouri|INFO| cp: (dd0b535a) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_dnase_avg_fseq_signal_formatted.txt.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/3b39284516e676ea52238f0636c0bbbf/hg38_dnase_avg_fseq_signal_formatted.txt.gz
2022-03-05 00:45:03,454|autouri.autouri|INFO| cp: (dd0b535a) skipped due to md5_match. md5=df624401f76fbd4d651e736068c43a1a
2022-03-05 00:45:03,458|autouri.autouri|INFO| cp: (7de7b933) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_celltype_compare_subsample.bed.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/c73f434c3fa4f3f54bc2ecad09c065c2/hg38_celltype_compare_subsample.bed.gz
2022-03-05 00:45:03,763|autouri.autouri|INFO| cp: (7de7b933) skipped due to md5_match. md5=ced0c653d28628654288f7a8ab052590
2022-03-05 00:45:03,766|autouri.autouri|INFO| cp: (4b14c116) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_dnase_avg_fseq_signal_metadata.txt, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/a9745b33b4ffdd83d7d2c5a7d3c8036a/hg38_dnase_avg_fseq_signal_metadata.txt
2022-03-05 00:45:03,994|autouri.autouri|INFO| cp: (4b14c116) skipped due to md5_match. md5=3f7fd85ab9a4c6274f28c3e82a79c10d
2022-03-05 00:45:04,045|autouri.autouri|INFO| cp: (9e879801) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/rep1.subsampled.25.fastq.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/41d3a8658bf4f565c08290e386fddccf/rep1.subsampled.25.fastq.gz
2022-03-05 00:45:04,487|autouri.autouri|INFO| cp: (9e879801) skipped due to md5_match. md5=212ac7a69357df3df077177b44c8cdf0
2022-03-05 00:45:04,492|autouri.autouri|INFO| cp: (1f07eb6a) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/rep2.subsampled.20.fastq.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/0d5b102d7f21dd009798ae9b2c318b65/rep2.subsampled.20.fastq.gz
2022-03-05 00:45:04,930|autouri.autouri|INFO| cp: (1f07eb6a) skipped due to md5_match. md5=4df1a7e6a3a299f6a2355d93b54db48a
2022-03-05 00:45:04,936|autouri.autouri|INFO| cp: (402c5ce4) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/ctl1.subsampled.25.fastq.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/f4dba75d4430f4702a75554c44265119/ctl1.subsampled.25.fastq.gz
2022-03-05 00:45:05,365|autouri.autouri|INFO| cp: (402c5ce4) skipped due to md5_match. md5=3ad89622e0b1269606575d305c5350ef
2022-03-05 00:45:05,370|autouri.autouri|INFO| cp: (0004b518) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-chip-seq-pipeline/ENCSR000DYI/fastq_subsampled/ctl2.subsampled.25.fastq.gz, dest=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/7a610dd5397c45a806678efaad824abf/ctl2.subsampled.25.fastq.gz
2022-03-05 00:45:05,785|autouri.autouri|INFO| cp: (0004b518) skipped due to md5_match. md5=b2a184523015df11edf3c670f07e1ed4
2022-03-05 00:45:07,153|caper.wdl_parser|ERROR| Failed to parse WDL with miniwdl.
2022-03-05 00:45:07,154|caper.caper_workflow_opts|INFO| Conda environment name not found in WDL metadata. wdl=/home/shirleytemples/chip-seq-pipeline2/chip.wdl
2022-03-05 00:45:07,179|caper.cromwell|INFO| Validating WDL/inputs/imports with Womtool...
2022-03-05 00:45:13,080|caper.cromwell|INFO| Passed Womtool validation.
2022-03-05 00:45:13,082|caper.caper_runner|INFO| launching run: wdl=/home/shirleytemples/chip-seq-pipeline2/chip.wdl, inputs=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/15b4ff439de58d859f4c3ee4482c3bd2/ENCSR000DYI_subsampled_chr19_only.local.json, backend_conf=/home/shirleytemples/chip-seq-pipeline2/.caper_tmp/chip/20220305_004455_192720/backend.conf
2022-03-05 00:45:27,348|caper.cromwell_workflow_monitor|INFO| Workflow: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, status=Submitted
2022-03-05 00:45:27,364|caper.cromwell_workflow_monitor|INFO| Workflow: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, status=Running
2022-03-05 00:45:42,010|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.read_genome_tsv:-1, retry=0, status=Started, job_id=13366
2022-03-05 00:45:42,015|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.read_genome_tsv:-1, retry=0, status=WaitingForReturnCode
2022-03-05 00:45:49,945|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.read_genome_tsv:-1, retry=0, status=Done
2022-03-05 00:46:01,988|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_ctl:1, retry=0, status=Started, job_id=13431
2022-03-05 00:46:01,990|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:0, retry=0, status=Started, job_id=13520
2022-03-05 00:46:01,992|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_ctl:1, retry=0, status=WaitingForReturnCode
2022-03-05 00:46:01,993|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_R1:0, retry=0, status=Started, job_id=13476
2022-03-05 00:46:01,995|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:0, retry=0, status=WaitingForReturnCode
2022-03-05 00:46:01,996|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_R1:0, retry=0, status=WaitingForReturnCode
2022-03-05 00:46:04,870|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_ctl:1, retry=0, status=Done
2022-03-05 00:46:06,983|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:1, retry=0, status=Started, job_id=13573
2022-03-05 00:46:06,983|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_ctl:0, retry=0, status=Started, job_id=13640
2022-03-05 00:46:06,989|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_ctl:0, retry=0, status=WaitingForReturnCode
2022-03-05 00:46:06,999|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:1, retry=0, status=WaitingForReturnCode
2022-03-05 00:46:07,319|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_R1:0, retry=0, status=Done
2022-03-05 00:46:08,495|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:0, retry=0, status=Done
2022-03-05 00:46:11,033|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:1, retry=0, status=Done
2022-03-05 00:46:11,997|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_R1:1, retry=0, status=Started, job_id=13709
2022-03-05 00:46:12,001|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_ctl:1, retry=1, status=Started, job_id=13776
2022-03-05 00:46:12,013|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_ctl:1, retry=1, status=WaitingForReturnCode
2022-03-05 00:46:12,016|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_R1:1, retry=0, status=WaitingForReturnCode
2022-03-05 00:46:12,629|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_ctl:0, retry=0, status=Done
2022-03-05 00:46:15,300|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_R1:1, retry=0, status=Done
2022-03-05 00:46:16,856|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_ctl:1, retry=1, status=Done
2022-03-05 00:46:16,988|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_R1:0, retry=1, status=Started, job_id=13878
2022-03-05 00:46:16,988|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:0, retry=1, status=Started, job_id=13945
2022-03-05 00:46:16,993|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_R1:0, retry=1, status=WaitingForReturnCode
2022-03-05 00:46:16,994|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:0, retry=1, status=WaitingForReturnCode
2022-03-05 00:46:21,026|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align_R1:0, retry=1, status=Done
2022-03-05 00:46:21,990|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:1, retry=1, status=Started, job_id=14018
2022-03-05 00:46:21,993|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:1, retry=1, status=WaitingForReturnCode
2022-03-05 00:46:24,279|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:1, retry=1, status=Done
2022-03-05 00:46:25,133|caper.cromwell_workflow_monitor|INFO| Task: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, task=chip.align:0, retry=1, status=Done
2022-03-05 00:46:26,133|caper.cromwell_workflow_monitor|INFO| Workflow: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, status=Failed
2022-03-05 00:46:38,336|caper.cromwell_metadata|INFO| Wrote metadata file. /home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/metadata.json
2022-03-05 00:46:38,336|caper.cromwell|INFO| Workflow failed. Auto-troubleshooting...
* Started troubleshooting workflow: id=e77e6cbd-4515-47b2-9fd5-f27506f041c2, status=Failed
* Found failures JSON object.
[
    {
        "message": "Workflow failed",
        "causedBy": [
            {
                "causedBy": [],
                "message": "Job chip.align_ctl:1:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details."
            },
            {
                "causedBy": [],
                "message": "Job chip.align_R1:0:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details."
            },
            {
                "causedBy": [],
                "message": "Job chip.align:1:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details."
            },
            {
                "causedBy": [],
                "message": "Job chip.align:0:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details."
            }
        ]
    }
]
* Recursively finding failures in calls (tasks)...

==== NAME=chip.align, STATUS=RetryableFailure, PARENT=
SHARD_IDX=0, RC=3, JOB_ID=13520
START=2022-03-05T08:46:01.194Z, END=2022-03-05T08:46:11.997Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/execution/*.bam': No such file or directory

==== NAME=chip.align, STATUS=Failed, PARENT=
SHARD_IDX=0, RC=3, JOB_ID=13945
START=2022-03-05T08:46:15.195Z, END=2022-03-05T08:46:25.139Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/attempt-2/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/attempt-2/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/attempt-2/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/attempt-2/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/attempt-2/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-0/attempt-2/execution/*.bam': No such file or directory

==== NAME=chip.align, STATUS=RetryableFailure, PARENT=
SHARD_IDX=1, RC=3, JOB_ID=13573
START=2022-03-05T08:46:03.197Z, END=2022-03-05T08:46:12.020Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/execution/*.bam': No such file or directory

==== NAME=chip.align, STATUS=Failed, PARENT=
SHARD_IDX=1, RC=3, JOB_ID=14018
START=2022-03-05T08:46:17.191Z, END=2022-03-05T08:46:24.284Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/attempt-2/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/attempt-2/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/attempt-2/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/attempt-2/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/attempt-2/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align/shard-1/attempt-2/execution/*.bam': No such file or directory

==== NAME=chip.align_ctl, STATUS=RetryableFailure, PARENT=
SHARD_IDX=0, RC=3, JOB_ID=13640
START=2022-03-05T08:46:05.195Z, END=2022-03-05T08:46:16.991Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-0/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-0/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-0/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-0/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-0/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-0/execution/*.bam': No such file or directory

==== NAME=chip.align_ctl, STATUS=RetryableFailure, PARENT=
SHARD_IDX=1, RC=3, JOB_ID=13431
START=2022-03-05T08:45:57.246Z, END=2022-03-05T08:46:06.993Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/execution/*.bam': No such file or directory

==== NAME=chip.align_ctl, STATUS=Failed, PARENT=
SHARD_IDX=1, RC=3, JOB_ID=13776
START=2022-03-05T08:46:09.194Z, END=2022-03-05T08:46:16.860Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/attempt-2/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/attempt-2/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/attempt-2/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/attempt-2/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/attempt-2/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_ctl/shard-1/attempt-2/execution/*.bam': No such file or directory

==== NAME=chip.align_R1, STATUS=RetryableFailure, PARENT=
SHARD_IDX=0, RC=3, JOB_ID=13476
START=2022-03-05T08:45:59.202Z, END=2022-03-05T08:46:11.997Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/execution/*.bam': No such file or directory

==== NAME=chip.align_R1, STATUS=Failed, PARENT=
SHARD_IDX=0, RC=3, JOB_ID=13878
START=2022-03-05T08:46:13.184Z, END=2022-03-05T08:46:21.031Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/attempt-2/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/attempt-2/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/attempt-2/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/attempt-2/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/attempt-2/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-0/attempt-2/execution/*.bam': No such file or directory

==== NAME=chip.align_R1, STATUS=RetryableFailure, PARENT=
SHARD_IDX=1, RC=3, JOB_ID=13709
START=2022-03-05T08:46:07.210Z, END=2022-03-05T08:46:16.990Z
STDOUT=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-1/execution/stdout
STDERR=/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-1/execution/stderr
STDERR_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.

STDERR_BACKGROUND_CONTENTS=

* Error: pipeline environment (docker, singularity or conda) not found.
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-1/execution/*.samstats.qc': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-1/execution/*.read_length.txt': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-1/execution/*.bai': No such file or directory
ln: failed to access '/home/shirleytemples/chip-seq-pipeline2/chip/e77e6cbd-4515-47b2-9fd5-f27506f041c2/call-align_R1/shard-1/execution/*.bam': No such file or directory

2022-03-05 00:46:38,486|caper.nb_subproc_thread|ERROR| Cromwell failed. returncode=1
2022-03-05 00:46:38,486|caper.cli|ERROR| Check stdout in /home/shirleytemples/chip-seq-pipeline2/cromwell.out.59
shirleytemples commented 2 years ago

The pipeline worked after I run the following: 

$ pip install caper --upgrade
# and then git pull (or git clone from scratch) the pipeline git directory to update it
$ scripts/uninstall_conda_env.sh
$ scripts/install_conda_env.sh