search

ENCODE-DCC / atac-seq-pipeline

ENCODE ATAC-seq pipeline
MIT License
383 stars 172 forks source link

Job exits with return code -1 #358

Open ionas23 opened 2 years ago

ionas23 commented 2 years ago

Dear team,

I am trying to run the ATAC seq pipeline but it encounters and error and exits in the first few minutes. My JSON file is correct and the genome.tsv URL is also correct, I have checked multiple times. This is the error:

2021-12-21 10:55:24,102|caper.cromwell|INFO| Workflow failed. Auto-troubleshooting...

==== NAME=atac.read_genome_tsv, STATUS=Failed, PARENT= SHARD_IDX=-1, RC=-1, JOB_ID=5518 START=2021-12-21T09:55:08.144Z, END=2021-12-21T09:55:13.354Z STDOUT=/staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/pipelineOutput/test2/atac/10b41fa4-25d5-4e52-9387-a382484de4b9/call-read_genome_tsv/execution/stdout STDERR=/staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/pipelineOutput/test2/atac/10b41fa4-25d5-4e52-9387-a382484de4b9/call-read_genome_tsv/execution/stderr STDERR_BACKGROUND_CONTENTS= /staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/pipelineOutput/test2/atac/10b41fa4-25d5-4e52-9387-a382484de4b9/call-read_genome_tsv/execution/script.submit: line 7: docker: command not found cat: /staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/pipelineOutput/test2/atac/10b41fa4-25d5-4e52-9387-a382484de4b9/call-read_genome_tsv/execution/docker_cid: No such file or directory /staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/pipelineOutput/test2/atac/10b41fa4-25d5-4e52-9387-a382484de4b9/call-read_genome_tsv/execution/script.submit: line 9: docker: command not found cat: /staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/pipelineOutput/test2/atac/10b41fa4-25d5-4e52-9387-a382484de4b9/call-read_genome_tsv/execution/docker_cid: No such file or directory /staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/pipelineOutput/test2/atac/10b41fa4-25d5-4e52-9387-a382484de4b9/call-read_genome_tsv/execution/script.submit: line 10: docker: command not found

2021-12-21 10:55:24,104|caper.nb_subproc_thread|ERROR| Cromwell failed. returncode=1 2021-12-21 10:55:24,104|caper.cli|ERROR| Check stdout in /lustre1/project/stg_00072/Ionas/test2/cromwell.out

OS/Platform

Caper configuration file

File: default.conf backend=local

local-hash-strat=path+modtime

local-loc-dir=

cromwell=/user/leuven/345/vsc34531/.caper/cromwell_jar/cromwell-65.jar womtool=/user/leuven/345/vsc34531/.caper/womtool_jar/womtool-65.jar

Input JSON file

{ "atac.title" : "Nelson_Robertson mTSC", "atac.description" : "Mapping the chromatin landscape and Blimp1 transcriptional targets that regulate trophoblast differentiation", "atac.paired_end" : true, "atac.genome_tsv" : "https://storage.googleapis.com/encode-pipeline-genome-data/genome_tsv/v3/mm10.tsv", "atac.fastqs_rep1_R1" : [ "/staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/Nelson_Robertson/TSC_line1_R1.fastq.gz"], "atac.fastqs_rep1_R2" : [ "/staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/Nelson_Robertson/TSC_line1_R2.fastq.gz"], "atac.fastqs_rep2_R1" : [ "/staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/Nelson_Robertson/TSC_line2_R1.fastq.gz"], "atac.fastqs_rep2_R2" : [ "/staging/leuven/stg_00072/Ionas/published_data/atacseq_analysis/Nelson_Robertson/TSC_line2_R2.fastq.gz"], "atac.auto_detect_adapter" : true, "atac.enable_gc_bias" : true, "atac.enable_tss_enrich" : true, "atac.enable_annot_enrich" : true, "atac.enable_jsd" : true,

"atac.align_cpu" : 10,
"atac.filter_cpu" : 10,
"atac.bam2ta_cpu" : 10,
"atac.call_peak_cpu" : 10

}

Troubleshooting result

Attached file cromwell.out.11.txt

leepc12 commented 2 years ago 
line 7: docker: command not found

Did you run caper run ... --docker on a computer without docker?

sermare commented 2 years ago

same problem here, adding the caper run .. --docker does not do anything

leepc12 commented 2 years ago

@sermare: Do you have docker on your system?

$ docker -v
Docker version 20.10.12, build 20.10.12-0ubuntu2~20.04.1

Please install Miniconda3 and follow instruction on README and see if that works.

sermare commented 2 years ago

I apologize. I have been using the conda environment, and running:

caper run ... --conda encd_atac

I am still getting a problem with my "task=atac.read_genome_tsv:-1"

leepc12 commented 2 years ago

Please try with --conda without the encd_atac part (it's actually encd-atac though and there are multiple environment for one pipeline.). Also, did you install pipeline's environment before running it?

$ conda env list | grep encd_atac
sermare commented 2 years ago

I ran without the encd_atac. It throws me an error:

Yes, I did the installing step using the bash scripts/install_conda_env.sh

Here is my cromwell.out output:

cromwell.txt

conda 4.12.0 OpenJDK Runtime Environment (build 11.0.1-internal+0-adhoc..src) OpenJDK 64-Bit Server VM (build 11.0.1-internal+0-adhoc..src, mixed mode) Python 3.6.6 :: Anaconda, Inc.

Here is my conda list: conda_list.txt

This is my config-file:

(encd-atac) [sermare@c4-log1 .caper]$ cat default.conf backend=local

Local directory for localized files and Cromwell's intermediate files. If not defined then Caper will make .caper_tmp/ on CWD orlocal-out-dir. /tmp is not recommended since Caper store localized data files here. local-loc-dir=

cromwell=/c4/home/sermare/.caper/cromwell_jar/cromwell-65.jar womtool=/c4/home/sermare/.caper/womtool_jar/womtool-65.jar

I had to update Java8 to Java11, because there was a Java error in the past. I hope this is normal too.

leepc12 commented 2 years ago

I wanted to see results of conda env list (not conda list). You should not run caper inside pipeline's Conda environment. Please install caper on your (base) environment and run it there.