search

ENCODE-DCC / atac-seq-pipeline

ENCODE ATAC-seq pipeline
MIT License
392 stars 174 forks source link

Conda installation errors #360

Open hariiyer16 opened 2 years ago

hariiyer16 commented 2 years ago

Describe the bug

I am trying to install the atac-seq-pipeline and having issues with installing the conda environments to even get the pipeline installed. Have been trying this for few weeks now but there have been issues. So I have done a clean install starting with fresh installation of Anaconda.

Following is what I have done:

conda config --set auto_activate_base false
#exit and come back 
source Anaconda3/etc/profile.d/conda.sh
conda activate base

cd ~/atac-seq-pipeline
$ bash scripts/uninstall_conda_env.sh  # uninstall it for clean-install
$ bash scripts/install_conda_env.sh mamba  # r

conda uninstall croo
pip install croo==0.3.4

OS/Platform

CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"

Caper configuration file

Paste contents of ~/.caper/default.conf.

backend=sge

# Parallel environement is required, ask your administrator to create one
# If your cluster doesn't support PE then edit 'sge-resource-param'
# to fit your cluster's configuration.
sge-pe=local

# This parameter is NOT for 'caper submit' BUT for 'caper run' and 'caper server' only.
# This resource parameter string will be passed to sbatch, qsub, bsub, ...
# You can customize it according to your cluster's configuration.

# Note that Cromwell's implicit type conversion (String to Integer)
# seems to be buggy for WomLong type memory variables (memory_mb and memory_gb).
# So be careful about using the + operator between WomLong and other types (String, even Int).
# For example, ${"--mem=" + memory_mb} will not work since memory_mb is WomLong.
# Use ${"if defined(memory_mb) then "--mem=" else ""}{memory_mb}${"if defined(memory_mb) then "mb " else " "}
# See https://github.com/broadinstitute/cromwell/issues/4659 for details

# Cromwell's built-in variables (attributes defined in WDL task's runtime)
# Use them within ${} notation.
# - cpu: number of cores for a job (default = 1)
# - memory_mb, memory_gb: total memory for a job in MB, GB
#   * these are converted from 'memory' string attribute (including size unit)
#     defined in WDL task's runtime
# - time: time limit for a job in hour
# - gpu: specified gpu name or number of gpus (it's declared as String)

# Parallel environment of SGE:
# Find one with `$ qconf -spl` or ask you admin to add one if not exists.
# If your cluster works without PE then edit the below sge-resource-param
sge-pe=local
sge-resource-param=${if cpu > 1 then "-pe " + sge_pe + " " else ""} ${if cpu > 1 then cpu else ""} ${true="-l h_vmem=$(expr " false="" defined(memory_mb)}${memory_mb}${true=" / " false="" defined(memory_mb)}${if defined(memory_mb) then cpu else ""}${true=")m" false="" defined(memory_mb)} ${true="-l s_vmem=$(expr " false="" defined(memory_mb)}${memory_mb}${true=" / " false="" defined(memory_mb)}${if defined(memory_mb) then cpu else ""}${true=")m" false="" defined(memory_mb)} ${"-l h_rt=" + time + ":00:00"} ${"-l s_rt=" + time + ":00:00"} ${"-l gpu=" + gpu}

# If needed uncomment and define any extra SGE qsub parameters here
# YOU CANNOT USE WDL SYNTAX AND CROMWELL BUILT-IN VARIABLES HERE
#sge-extra-param=

# Hashing strategy for call-caching (3 choices)
# This parameter is for local (local/slurm/sge/pbs/lsf) backend only.
# This is important for call-caching,
# which means re-using outputs from previous/failed workflows.
# Cache will miss if different strategy is used.
# "file" method has been default for all old versions of Caper<1.0.
# "path+modtime" is a new default for Caper>=1.0,
#   file: use md5sum hash (slow).
#   path: use path.
#   path+modtime: use path and modification time.
local-hash-strat=path+modtime

# Metadata DB for call-caching (reusing previous outputs):
# Cromwell supports restarting workflows based on a metadata DB
# DB is in-memory by default
#db=in-memory

# If you use 'caper server' then you can use one unified '--file-db'
# for all submitted workflows. In such case, uncomment the following two lines
# and defined file-db as an absolute path to store metadata of all workflows
#db=file
#file-db=

# If you use 'caper run' and want to use call-caching:
# Make sure to define different 'caper run ... --db file --file-db DB_PATH'
# for each pipeline run.
# But if you want to restart then define the same '--db file --file-db DB_PATH'
# then Caper will collect/re-use previous outputs without running the same task again
# Previous outputs will be simply hard/soft-linked.

# Local directory for localized files and Cromwell's intermediate files
# If not defined, Caper will make .caper_tmp/ on local-out-dir or CWD.
# /tmp is not recommended here since Caper store all localized data files
# on this directory (e.g. input FASTQs defined as URLs in input JSON).
local-loc-dir=

cromwell=/users/heaswara/.caper/cromwell_jar/cromwell-65.jar
womtool=/users/heaswara/.caper/womtool_jar/womtool-65.jar

When I try running the example caper command, i get the following error.

$ caper run atac.wdl -i https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ_subsampled.json --conda
Traceback (most recent call last):
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/arg_tool.py", line 42, in read_from_conf
    config.read_string(s)
  File "/users/heaswara/Anaconda3/lib/python3.9/configparser.py", line 723, in read_string
    self.read_file(sfile, source)
  File "/users/heaswara/Anaconda3/lib/python3.9/configparser.py", line 718, in read_file
    self._read(f, source)
  File "/users/heaswara/Anaconda3/lib/python3.9/configparser.py", line 1085, in _read
    raise MissingSectionHeaderError(fpname, lineno, line)
configparser.MissingSectionHeaderError: File contains no section headers.
file: '<string>', line: 2
'backend=sge\n'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/users/heaswara/Anaconda3/bin/caper", line 13, in <module>
    main()
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/cli.py", line 683, in main
    parser, _ = get_parser_and_defaults()
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/caper_args.py", line 886, in get_parser_and_defaults
    conf_dict = update_parsers_defaults_with_conf(
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/arg_tool.py", line 116, in update_parsers_defaults_with_conf
    defaults = read_from_conf(
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/arg_tool.py", line 45, in read_from_conf
    config.read_string(section + s)
  File "/users/heaswara/Anaconda3/lib/python3.9/configparser.py", line 723, in read_string
    self.read_file(sfile, source)
  File "/users/heaswara/Anaconda3/lib/python3.9/configparser.py", line 718, in read_file
    self._read(f, source)
  File "/users/heaswara/Anaconda3/lib/python3.9/configparser.py", line 1096, in _read
    raise DuplicateOptionError(sectname, optname,
configparser.DuplicateOptionError: While reading from '<string>' [line 34]: option 'sge-pe' in section 'defaults' already exists

Any help is gratefully appreciated as have been trying this for a long time now. Thank you.

leepc12 commented 2 years ago

You have a duplicate item (sge-pe=local) in the conf file.

hariiyer16 commented 2 years ago

Thank you so much. I noticed that while again trying and removed the second instance. Apparently caper init automatically generates two instances of '''sge-pe'''. I got rid of the 2nd instance. And ... voilà this started to work... caper run atac.wdl -i https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ_subsampled.json --conda encode-atac-seq-pipeline. But....now I get teh following java error.

Traceback (most recent call last): File "/users/heaswara/Anaconda3/bin/caper", line 13, in <module> main() File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/cli.py", line 705, in main return runner(parsed_args, nonblocking_server=nonblocking_server) File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/cli.py", line 249, in runner subcmd_run(c, args) File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/cli.py", line 379, in subcmd_run thread = caper_runner.run( File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/caper_runner.py", line 462, in run self._cromwell.validate(wdl=wdl, inputs=inputs, imports=imports) File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/cromwell.py", line 154, in validate raise WomtoolValidationFailed( caper.cromwell.WomtoolValidationFailed: RC=1 STDERR=Exception in thread "main" java.lang.BootstrapMethodError: java.lang.UnsupportedClassVersionError: wdl/draft3/parser/WdlParser$Ast has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0

hariiyer16 commented 2 years ago

Hi Jin, full output of running caper run atac.wdl -i https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ_subsampled.json --conda encode-atac-seq-pipeline is shown below:

caper run atac.wdl -i https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ_subsampled.json --conda encode-atac-seq-pipeline 2022-01-06 15:05:04,216|caper.cli|INFO| Cromwell stdout: /users/heaswara/atac-seq-pipeline/cromwell.out 2022-01-06 15:05:04,230|caper.caper_base|INFO| Creating a timestamped temporary directory. /legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/atac/20220106_150504_219604 2022-01-06 15:05:04,230|caper.caper_runner|INFO| Localizing files on work_dir. /legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/atac/20220106_150504_219604 2022-01-06 15:05:04,493|autouri.autouri|INFO| cp: (6b6312ef) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/caf534ed3cf684406e731d19be272b4a/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.gz 2022-01-06 15:05:41,838|autouri.autouri|INFO| cp: (6b6312ef) done. 2022-01-06 15:05:41,839|autouri.autouri|INFO| cp: (35403bb4) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only.fasta.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/f43b63a83784d3ec8055f1a22168ed89/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only.fasta.gz 2022-01-06 15:05:42,594|autouri.autouri|INFO| cp: (35403bb4) done. 2022-01-06 15:05:42,595|autouri.autouri|INFO| cp: (fc074f2d) started. src=https://www.encodeproject.org/files/ENCFF356LFX/@@download/ENCFF356LFX.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/f183dcba5d34f959d8b55ed438ee2e22/ENCFF356LFX.bed.gz 2022-01-06 15:05:43,343|autouri.autouri|INFO| cp: (fc074f2d) done. 2022-01-06 15:05:44,196|autouri.autouri|INFO| cp: (2835409f) started. src=https://www.encodeproject.org/files/GRCh38_EBV.chrom.sizes/@@download/GRCh38_EBV.chrom.sizes.tsv, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/c52f52c7bfa357f55a39b1de7e4d0b0c/GRCh38_EBV.chrom.sizes.tsv 2022-01-06 15:05:44,943|autouri.autouri|INFO| cp: (2835409f) done. 2022-01-06 15:05:44,944|autouri.autouri|INFO| cp: (839d8686) started. src=https://www.encodeproject.org/files/ENCFF110MCL/@@download/ENCFF110MCL.tar.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/3ff4ac4c3f59d096b1a3842a182072ae/ENCFF110MCL.tar.gz 2022-01-06 15:08:36,670|autouri.autouri|INFO| cp: (839d8686) done. 2022-01-06 15:08:36,671|autouri.autouri|INFO| cp: (41cae902) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/df5193e07055d13c48be59bacd0f56b8/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz 2022-01-06 15:08:37,635|autouri.autouri|INFO| cp: (41cae902) done. 2022-01-06 15:08:37,636|autouri.autouri|INFO| cp: (e74417dc) started. src=https://www.encodeproject.org/files/ENCFF643CGH/@@download/ENCFF643CGH.tar.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/8c692fba4640609720272154ab0faa30/ENCFF643CGH.tar.gz 2022-01-06 15:12:33,837|autouri.autouri|INFO| cp: (e74417dc) done. 2022-01-06 15:12:33,838|autouri.autouri|INFO| cp: (6330d24a) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bwa_index/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bwa_index.tar.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/d3dff25534e93d893902540d81e4f475/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bwa_index.tar.gz 2022-01-06 15:12:34,735|autouri.autouri|INFO| cp: (6330d24a) done. 2022-01-06 15:12:34,736|autouri.autouri|INFO| cp: (3735e5c8) started. src=https://www.encodeproject.org/files/ENCFF493CCB/@@download/ENCFF493CCB.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/0fa7d04b32e66fa02fb2c1ae39e41447/ENCFF493CCB.bed.gz 2022-01-06 15:12:35,750|autouri.autouri|INFO| cp: (3735e5c8) done. 2022-01-06 15:12:35,751|autouri.autouri|INFO| cp: (875f31c8) started. src=https://www.encodeproject.org/files/ENCFF304XEX/@@download/ENCFF304XEX.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/805e179275a9c0fb7a37def40c4312d1/ENCFF304XEX.bed.gz 2022-01-06 15:12:38,079|autouri.autouri|INFO| cp: (875f31c8) done. 2022-01-06 15:12:38,080|autouri.autouri|INFO| cp: (e2ad8a09) started. src=https://www.encodeproject.org/files/ENCFF140XLU/@@download/ENCFF140XLU.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/0cbd2c602ddad252bc39729fc8a29286/ENCFF140XLU.bed.gz 2022-01-06 15:12:40,124|autouri.autouri|INFO| cp: (e2ad8a09) done. 2022-01-06 15:12:40,125|autouri.autouri|INFO| cp: (55c7cf56) started. src=https://www.encodeproject.org/files/ENCFF212UAV/@@download/ENCFF212UAV.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/1d3aa436b05f16a509edb94789c061d3/ENCFF212UAV.bed.gz 2022-01-06 15:12:53,261|autouri.autouri|INFO| cp: (55c7cf56) done. 2022-01-06 15:12:53,262|autouri.autouri|INFO| cp: (684a9f25) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_dnase_avg_fseq_signal_formatted.txt.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/3b39284516e676ea52238f0636c0bbbf/hg38_dnase_avg_fseq_signal_formatted.txt.gz 2022-01-06 15:12:56,415|autouri.autouri|INFO| cp: (684a9f25) done. 2022-01-06 15:12:56,416|autouri.autouri|INFO| cp: (dc122341) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_celltype_compare_subsample.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/c73f434c3fa4f3f54bc2ecad09c065c2/hg38_celltype_compare_subsample.bed.gz 2022-01-06 15:12:56,567|autouri.autouri|INFO| cp: (dc122341) done. 2022-01-06 15:12:56,568|autouri.autouri|INFO| cp: (2c646f37) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_dnase_avg_fseq_signal_metadata.txt, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/a9745b33b4ffdd83d7d2c5a7d3c8036a/hg38_dnase_avg_fseq_signal_metadata.txt 2022-01-06 15:12:56,671|autouri.autouri|INFO| cp: (2c646f37) done. 2022-01-06 15:12:56,701|autouri.autouri|INFO| cp: (9993fc7d) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair1/ENCFF341MYG.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/7ded2935aace2939d00d8fb5b47867ae/ENCFF341MYG.subsampled.400.fastq.gz 2022-01-06 15:12:57,057|autouri.autouri|INFO| cp: (9993fc7d) done. 2022-01-06 15:12:57,058|autouri.autouri|INFO| cp: (c414f0b2) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair1/ENCFF106QGY.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/ca04bcf10cb8ac66068bc5f15a436f03/ENCFF106QGY.subsampled.400.fastq.gz 2022-01-06 15:12:57,607|autouri.autouri|INFO| cp: (c414f0b2) done. 2022-01-06 15:12:57,607|autouri.autouri|INFO| cp: (49bb24db) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair2/ENCFF248EJF.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/27d48f46ef8e2ac6910ac87f3ccbe163/ENCFF248EJF.subsampled.400.fastq.gz 2022-01-06 15:12:58,212|autouri.autouri|INFO| cp: (49bb24db) done. 2022-01-06 15:12:58,212|autouri.autouri|INFO| cp: (35c15b92) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair2/ENCFF368TYI.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/a7ff3728754970409a3d2f093cf38b52/ENCFF368TYI.subsampled.400.fastq.gz 2022-01-06 15:12:58,988|autouri.autouri|INFO| cp: (35c15b92) done. 2022-01-06 15:12:58,989|autouri.autouri|INFO| cp: (ef7dfa6e) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF641SFZ.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/3c6d0cab14c16eaeaf22560de63ac804/ENCFF641SFZ.subsampled.400.fastq.gz 2022-01-06 15:12:59,276|autouri.autouri|INFO| cp: (ef7dfa6e) done. 2022-01-06 15:12:59,277|autouri.autouri|INFO| cp: (cc6bffae) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF751XTV.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/218fa2f732797e84fa153d479e7dcafb/ENCFF751XTV.subsampled.400.fastq.gz 2022-01-06 15:12:59,569|autouri.autouri|INFO| cp: (cc6bffae) done. 2022-01-06 15:12:59,570|autouri.autouri|INFO| cp: (29142460) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF927LSG.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/dfea21b33ce8e0ab13fd158e7e812586/ENCFF927LSG.subsampled.400.fastq.gz 2022-01-06 15:12:59,920|autouri.autouri|INFO| cp: (29142460) done. 2022-01-06 15:12:59,921|autouri.autouri|INFO| cp: (7d1703e5) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF859BDM.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/44b253b6db9e0ea8de62aea604878556/ENCFF859BDM.subsampled.400.fastq.gz 2022-01-06 15:13:00,548|autouri.autouri|INFO| cp: (7d1703e5) done. 2022-01-06 15:13:00,549|autouri.autouri|INFO| cp: (35dfaf6e) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF193RRC.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/db6282276790c12cb2e76b1ed84fb26f/ENCFF193RRC.subsampled.400.fastq.gz 2022-01-06 15:13:00,985|autouri.autouri|INFO| cp: (35dfaf6e) done. 2022-01-06 15:13:00,986|autouri.autouri|INFO| cp: (508a3ada) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF366DFI.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/88672b6fafd695007baf55f7f9757fed/ENCFF366DFI.subsampled.400.fastq.gz 2022-01-06 15:13:01,494|autouri.autouri|INFO| cp: (508a3ada) done. 2022-01-06 15:13:01,495|autouri.autouri|INFO| cp: (8c0a9689) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF031ARQ.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/713c163a0f65d7350956d72323fc0310/ENCFF031ARQ.subsampled.400.fastq.gz 2022-01-06 15:13:01,846|autouri.autouri|INFO| cp: (8c0a9689) done. 2022-01-06 15:13:01,847|autouri.autouri|INFO| cp: (5176e5af) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF590SYZ.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/1654703543de9eb53b95469e4c26856c/ENCFF590SYZ.subsampled.400.fastq.gz 2022-01-06 15:13:02,201|autouri.autouri|INFO| cp: (5176e5af) done. 2022-01-06 15:13:02,202|autouri.autouri|INFO| cp: (e74d06d3) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF734PEQ.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/69b00bab505645c65eb5957a0b6b2392/ENCFF734PEQ.subsampled.400.fastq.gz 2022-01-06 15:13:02,561|autouri.autouri|INFO| cp: (e74d06d3) done. 2022-01-06 15:13:02,562|autouri.autouri|INFO| cp: (23ce5061) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF007USV.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/3810785b7e92ca7d6c4612c20c3ca534/ENCFF007USV.subsampled.400.fastq.gz 2022-01-06 15:13:02,895|autouri.autouri|INFO| cp: (23ce5061) done. 2022-01-06 15:13:02,896|autouri.autouri|INFO| cp: (f14038d3) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF886FSC.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/482c9ce8ed007c251adca385c7d4ce95/ENCFF886FSC.subsampled.400.fastq.gz 2022-01-06 15:13:03,180|autouri.autouri|INFO| cp: (f14038d3) done. 2022-01-06 15:13:03,181|autouri.autouri|INFO| cp: (539fabf7) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF573UXK.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/a3a22d635ec162098e31bcea6655fb34/ENCFF573UXK.subsampled.400.fastq.gz 2022-01-06 15:13:03,753|autouri.autouri|INFO| cp: (539fabf7) done. 2022-01-06 15:13:06,538|caper.cromwell|INFO| Validating WDL/inputs/imports with Womtool... Traceback (most recent call last): File "/users/heaswara/Anaconda3/bin/caper", line 13, in <module> main() File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/cli.py", line 705, in main return runner(parsed_args, nonblocking_server=nonblocking_server) File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/cli.py", line 249, in runner subcmd_run(c, args) File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/cli.py", line 379, in subcmd_run thread = caper_runner.run( File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/caper_runner.py", line 462, in run self._cromwell.validate(wdl=wdl, inputs=inputs, imports=imports) File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/caper/cromwell.py", line 154, in validate raise WomtoolValidationFailed( caper.cromwell.WomtoolValidationFailed: RC=1 STDERR=Exception in thread "main" java.lang.BootstrapMethodError: java.lang.UnsupportedClassVersionError: wdl/draft3/parser/WdlParser$Ast has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0 at wdl.draft3.transforms.ast2wdlom.package$.<init>(ast2wdlom.scala:16) at wdl.draft3.transforms.ast2wdlom.package$.<clinit>(ast2wdlom.scala) at languages.wdl.draft3.WdlDraft3LanguageFactory.makeWomBundle(WdlDraft3LanguageFactory.scala:64) at languages.wdl.draft3.WdlDraft3LanguageFactory$$anon$1.call(WdlDraft3LanguageFactory.scala:79) at languages.wdl.draft3.WdlDraft3LanguageFactory$$anon$1.call(WdlDraft3LanguageFactory.scala:78) at cromwell.languages.util.ParserCache.$anonfun$retrieveOrCalculate$2(ParserCache.scala:35) at scala.Option.getOrElse(Option.scala:189) at cromwell.languages.util.ParserCache.retrieveOrCalculate(ParserCache.scala:35) at cromwell.languages.util.ParserCache.retrieveOrCalculate$(ParserCache.scala:25) at languages.wdl.draft3.WdlDraft3LanguageFactory.retrieveOrCalculate(WdlDraft3LanguageFactory.scala:30) at languages.wdl.draft3.WdlDraft3LanguageFactory.$anonfun$getWomBundle$2(WdlDraft3LanguageFactory.scala:86) at scala.util.Either.flatMap(Either.scala:341) at languages.wdl.draft3.WdlDraft3LanguageFactory.getWomBundle(WdlDraft3LanguageFactory.scala:85) at womtool.input.WomGraphMaker$.$anonfun$getBundleAndFactory$1(WomGraphMaker.scala:40) at scala.util.Either.flatMap(Either.scala:341) at womtool.input.WomGraphMaker$.getBundleAndFactory(WomGraphMaker.scala:31) at womtool.input.WomGraphMaker$.fromFiles(WomGraphMaker.scala:47) at womtool.validate.Validate$.validate(Validate.scala:26) at womtool.WomtoolMain$.dispatchCommand(WomtoolMain.scala:54) at womtool.WomtoolMain$.runWomtool(WomtoolMain.scala:161) at womtool.WomtoolMain$.delayedEndpoint$womtool$WomtoolMain$1(WomtoolMain.scala:166) at womtool.WomtoolMain$delayedInit$body.apply(WomtoolMain.scala:27) at scala.Function0.apply$mcV$sp(Function0.scala:39) at scala.Function0.apply$mcV$sp$(Function0.scala:39) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17) at scala.App.$anonfun$main$1$adapted(App.scala:80) at scala.collection.immutable.List.foreach(List.scala:431) at scala.App.main(App.scala:80) at scala.App.main$(App.scala:78) at womtool.WomtoolMain$.main(WomtoolMain.scala:27) at womtool.WomtoolMain.main(WomtoolMain.scala) Caused by: java.lang.UnsupportedClassVersionError: wdl/draft3/parser/WdlParser$Ast has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:756) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at java.net.URLClassLoader.access$100(URLClassLoader.java:74) at java.net.URLClassLoader$1.run(URLClassLoader.java:369) at java.net.URLClassLoader$1.run(URLClassLoader.java:363) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:362) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 31 more

Thank you so much again for your help. I hope to get this working.

leepc12 commented 2 years ago

Upgrade your system's java >= 11.

$ java -version
Picked up _JAVA_OPTIONS: -Xms256M -Xmx4024M -XX:ParallelGCThreads=1
openjdk version "11.0.13" 2021-10-19
OpenJDK Runtime Environment (build 11.0.13+8-Ubuntu-0ubuntu1.18.04)
OpenJDK 64-Bit Server VM (build 11.0.13+8-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)
hariiyer16 commented 2 years ago

Thanks Jin. Will check the java upgrade. Winder if we need update via the system admin in the cluster. Will keep you posted.

leepc12 commented 2 years ago

I think you can locally install java 11 and add it to PATH env var in your ~/.bashrc. e.g. export PATH=.......:$PATH.

hariiyer16 commented 2 years ago

Let me check that. 👍🏼

hariiyer16 commented 2 years ago

Hi jin. Managed to get java >11. I am trying the example run (ENCSR356KRQ_subsampled.json). How long does it typically take for the run to complete. To me it appears it is stuck at one process WaitingForReturnCode (please see the output below). In a previous run, it was similarly stuck at one step for more than 2 days.... Just wondering if this is normal. $ caper run atac.wdl -i https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ_subsampled.json --conda atac-seq-pipeline 2022-01-07 10:19:55,607|caper.cli|INFO| Cromwell stdout: /users/heaswara/atac-seq-pipeline/cromwell.out.2 2022-01-07 10:19:55,618|caper.caper_base|INFO| Creating a timestamped temporary directory. /legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/atac/20220107_101955_613311 2022-01-07 10:19:55,618|caper.caper_runner|INFO| Localizing files on work_dir. /legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/atac/20220107_101955_613311 2022-01-07 10:19:55,905|autouri.autouri|INFO| cp: (a553aced) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/caf534ed3cf684406e731d19be272b4a/GRCh38_no_alt_analysis_set_GCA_000001405.15.fasta.gz 2022-01-07 10:20:31,500|autouri.autouri|INFO| cp: (a553aced) done. 2022-01-07 10:20:31,503|autouri.autouri|INFO| cp: (60fdd41a) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only.fasta.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/f43b63a83784d3ec8055f1a22168ed89/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only.fasta.gz 2022-01-07 10:20:32,273|autouri.autouri|INFO| cp: (60fdd41a) done. 2022-01-07 10:20:32,275|autouri.autouri|INFO| cp: (8cef2edf) started. src=https://www.encodeproject.org/files/ENCFF356LFX/@@download/ENCFF356LFX.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/f183dcba5d34f959d8b55ed438ee2e22/ENCFF356LFX.bed.gz 2022-01-07 10:20:32,989|autouri.autouri|INFO| cp: (8cef2edf) done. 2022-01-07 10:20:33,794|autouri.autouri|INFO| cp: (8f961a76) started. src=https://www.encodeproject.org/files/GRCh38_EBV.chrom.sizes/@@download/GRCh38_EBV.chrom.sizes.tsv, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/c52f52c7bfa357f55a39b1de7e4d0b0c/GRCh38_EBV.chrom.sizes.tsv 2022-01-07 10:20:34,507|autouri.autouri|INFO| cp: (8f961a76) done. 2022-01-07 10:20:34,509|autouri.autouri|INFO| cp: (791c48cd) started. src=https://www.encodeproject.org/files/ENCFF110MCL/@@download/ENCFF110MCL.tar.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/3ff4ac4c3f59d096b1a3842a182072ae/ENCFF110MCL.tar.gz 2022-01-07 10:23:09,507|autouri.autouri|INFO| cp: (791c48cd) done. 2022-01-07 10:23:09,508|autouri.autouri|INFO| cp: (30d0c263) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/df5193e07055d13c48be59bacd0f56b8/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bowtie2_index.tar.gz 2022-01-07 10:23:10,605|autouri.autouri|INFO| cp: (30d0c263) done. 2022-01-07 10:23:10,606|autouri.autouri|INFO| cp: (bebbde54) started. src=https://www.encodeproject.org/files/ENCFF643CGH/@@download/ENCFF643CGH.tar.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/8c692fba4640609720272154ab0faa30/ENCFF643CGH.tar.gz 2022-01-07 10:26:23,941|autouri.autouri|INFO| cp: (bebbde54) done. 2022-01-07 10:26:23,942|autouri.autouri|INFO| cp: (96c1ee15) started. src=https://www.encodeproject.org/files/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bwa_index/@@download/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bwa_index.tar.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/d3dff25534e93d893902540d81e4f475/GRCh38_no_alt_analysis_set_GCA_000001405.15_mito_only_bwa_index.tar.gz 2022-01-07 10:26:24,892|autouri.autouri|INFO| cp: (96c1ee15) done. 2022-01-07 10:26:24,893|autouri.autouri|INFO| cp: (b5f26690) started. src=https://www.encodeproject.org/files/ENCFF766FGL/@@download/ENCFF766FGL.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/f16cc7892540d7a9bc3207cc44eb8288/ENCFF766FGL.bed.gz 2022-01-07 10:26:25,906|autouri.autouri|INFO| cp: (b5f26690) done. 2022-01-07 10:26:25,907|autouri.autouri|INFO| cp: (52671930) started. src=https://www.encodeproject.org/files/ENCFF304XEX/@@download/ENCFF304XEX.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/805e179275a9c0fb7a37def40c4312d1/ENCFF304XEX.bed.gz 2022-01-07 10:26:27,595|autouri.autouri|INFO| cp: (52671930) done. 2022-01-07 10:26:27,596|autouri.autouri|INFO| cp: (89064799) started. src=https://www.encodeproject.org/files/ENCFF140XLU/@@download/ENCFF140XLU.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/0cbd2c602ddad252bc39729fc8a29286/ENCFF140XLU.bed.gz 2022-01-07 10:26:28,793|autouri.autouri|INFO| cp: (89064799) done. 2022-01-07 10:26:28,794|autouri.autouri|INFO| cp: (50c23a9e) started. src=https://www.encodeproject.org/files/ENCFF212UAV/@@download/ENCFF212UAV.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/1d3aa436b05f16a509edb94789c061d3/ENCFF212UAV.bed.gz 2022-01-07 10:26:30,650|autouri.autouri|INFO| cp: (50c23a9e) done. 2022-01-07 10:26:30,651|autouri.autouri|INFO| cp: (d4aac771) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_dnase_avg_fseq_signal_formatted.txt.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/3b39284516e676ea52238f0636c0bbbf/hg38_dnase_avg_fseq_signal_formatted.txt.gz 2022-01-07 10:26:32,576|autouri.autouri|INFO| cp: (d4aac771) done. 2022-01-07 10:26:32,578|autouri.autouri|INFO| cp: (9109e0da) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_celltype_compare_subsample.bed.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/c73f434c3fa4f3f54bc2ecad09c065c2/hg38_celltype_compare_subsample.bed.gz 2022-01-07 10:26:32,831|autouri.autouri|INFO| cp: (9109e0da) done. 2022-01-07 10:26:32,832|autouri.autouri|INFO| cp: (b2d9e7bd) started. src=https://storage.googleapis.com/encode-pipeline-genome-data/hg38/ataqc/hg38_dnase_avg_fseq_signal_metadata.txt, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/a9745b33b4ffdd83d7d2c5a7d3c8036a/hg38_dnase_avg_fseq_signal_metadata.txt 2022-01-07 10:26:32,912|autouri.autouri|INFO| cp: (b2d9e7bd) done. 2022-01-07 10:26:32,937|autouri.autouri|INFO| cp: (e8fbd76d) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair1/ENCFF341MYG.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/7ded2935aace2939d00d8fb5b47867ae/ENCFF341MYG.subsampled.400.fastq.gz 2022-01-07 10:26:33,663|autouri.autouri|INFO| cp: (e8fbd76d) done. 2022-01-07 10:26:33,664|autouri.autouri|INFO| cp: (2e9e29da) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair1/ENCFF106QGY.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/ca04bcf10cb8ac66068bc5f15a436f03/ENCFF106QGY.subsampled.400.fastq.gz 2022-01-07 10:26:33,961|autouri.autouri|INFO| cp: (2e9e29da) done. 2022-01-07 10:26:33,962|autouri.autouri|INFO| cp: (7d30a33e) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair2/ENCFF248EJF.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/27d48f46ef8e2ac6910ac87f3ccbe163/ENCFF248EJF.subsampled.400.fastq.gz 2022-01-07 10:26:34,524|autouri.autouri|INFO| cp: (7d30a33e) done. 2022-01-07 10:26:34,525|autouri.autouri|INFO| cp: (0f2fbfe3) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep1/pair2/ENCFF368TYI.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/a7ff3728754970409a3d2f093cf38b52/ENCFF368TYI.subsampled.400.fastq.gz 2022-01-07 10:26:34,951|autouri.autouri|INFO| cp: (0f2fbfe3) done. 2022-01-07 10:26:34,952|autouri.autouri|INFO| cp: (2e54b933) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF641SFZ.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/3c6d0cab14c16eaeaf22560de63ac804/ENCFF641SFZ.subsampled.400.fastq.gz 2022-01-07 10:26:35,199|autouri.autouri|INFO| cp: (2e54b933) done. 2022-01-07 10:26:35,200|autouri.autouri|INFO| cp: (773319b8) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF751XTV.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/218fa2f732797e84fa153d479e7dcafb/ENCFF751XTV.subsampled.400.fastq.gz 2022-01-07 10:26:35,399|autouri.autouri|INFO| cp: (773319b8) done. 2022-01-07 10:26:35,400|autouri.autouri|INFO| cp: (25a3aea7) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF927LSG.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/dfea21b33ce8e0ab13fd158e7e812586/ENCFF927LSG.subsampled.400.fastq.gz 2022-01-07 10:26:35,653|autouri.autouri|INFO| cp: (25a3aea7) done. 2022-01-07 10:26:35,654|autouri.autouri|INFO| cp: (e84cd81c) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF859BDM.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/44b253b6db9e0ea8de62aea604878556/ENCFF859BDM.subsampled.400.fastq.gz 2022-01-07 10:26:35,869|autouri.autouri|INFO| cp: (e84cd81c) done. 2022-01-07 10:26:35,870|autouri.autouri|INFO| cp: (16a7cfe8) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF193RRC.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/db6282276790c12cb2e76b1ed84fb26f/ENCFF193RRC.subsampled.400.fastq.gz 2022-01-07 10:26:36,166|autouri.autouri|INFO| cp: (16a7cfe8) done. 2022-01-07 10:26:36,167|autouri.autouri|INFO| cp: (d418a4f2) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair1/ENCFF366DFI.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/88672b6fafd695007baf55f7f9757fed/ENCFF366DFI.subsampled.400.fastq.gz 2022-01-07 10:26:36,302|autouri.autouri|INFO| cp: (d418a4f2) done. 2022-01-07 10:26:36,303|autouri.autouri|INFO| cp: (deb45297) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF031ARQ.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/713c163a0f65d7350956d72323fc0310/ENCFF031ARQ.subsampled.400.fastq.gz 2022-01-07 10:26:36,586|autouri.autouri|INFO| cp: (deb45297) done. 2022-01-07 10:26:36,587|autouri.autouri|INFO| cp: (07370542) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF590SYZ.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/1654703543de9eb53b95469e4c26856c/ENCFF590SYZ.subsampled.400.fastq.gz 2022-01-07 10:26:36,706|autouri.autouri|INFO| cp: (07370542) done. 2022-01-07 10:26:36,707|autouri.autouri|INFO| cp: (68cf08f6) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF734PEQ.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/69b00bab505645c65eb5957a0b6b2392/ENCFF734PEQ.subsampled.400.fastq.gz 2022-01-07 10:26:36,948|autouri.autouri|INFO| cp: (68cf08f6) done. 2022-01-07 10:26:36,949|autouri.autouri|INFO| cp: (c95b92b9) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF007USV.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/3810785b7e92ca7d6c4612c20c3ca534/ENCFF007USV.subsampled.400.fastq.gz 2022-01-07 10:26:37,115|autouri.autouri|INFO| cp: (c95b92b9) done. 2022-01-07 10:26:37,116|autouri.autouri|INFO| cp: (a3893232) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF886FSC.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/482c9ce8ed007c251adca385c7d4ce95/ENCFF886FSC.subsampled.400.fastq.gz 2022-01-07 10:26:37,366|autouri.autouri|INFO| cp: (a3893232) done. 2022-01-07 10:26:37,367|autouri.autouri|INFO| cp: (ae584ff1) started. src=https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ/fastq_subsampled/rep2/pair2/ENCFF573UXK.subsampled.400.fastq.gz, dest=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/a3a22d635ec162098e31bcea6655fb34/ENCFF573UXK.subsampled.400.fastq.gz 2022-01-07 10:26:37,658|autouri.autouri|INFO| cp: (ae584ff1) done. 2022-01-07 10:26:40,215|caper.cromwell|INFO| Validating WDL/inputs/imports with Womtool... 2022-01-07 10:26:59,149|caper.cromwell|INFO| Passed Womtool validation. 2022-01-07 10:26:59,150|caper.caper_runner|INFO| launching run: wdl=/users/heaswara/atac-seq-pipeline/atac.wdl, inputs=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/4b5ea65d4461c146c96a3239d93d05ce/ENCSR356KRQ_subsampled.local.json, backend_conf=/legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/caper_temp/atac/20220107_101955_613311/backend.conf 2022-01-07 10:27:41,656|caper.cromwell_workflow_monitor|INFO| Workflow: id=4924782e-b9fc-4ac7-b17a-04264216a4d9, status=Submitted 2022-01-07 10:27:41,930|caper.cromwell_workflow_monitor|INFO| Workflow: id=4924782e-b9fc-4ac7-b17a-04264216a4d9, status=Running 2022-01-07 10:28:09,715|caper.cromwell_workflow_monitor|INFO| Task: id=4924782e-b9fc-4ac7-b17a-04264216a4d9, task=atac.read_genome_tsv:-1, retry=0, status=Started, job_id=7081 2022-01-07 10:28:09,750|caper.cromwell_workflow_monitor|INFO| Task: id=4924782e-b9fc-4ac7-b17a-04264216a4d9, task=atac.read_genome_tsv:-1, retry=0, status=WaitingForReturnCode

leepc12 commented 2 years ago

If it is stuck at atac.read_genome_tsv then something is wrong. This task should be done in a minute.

Please use the latest pipeline. I found this in your comment. This means that you are using an outdated pipeline.

$ bash scripts/install_conda_env.sh mamba  # <-------------------- this doesn't work with the latest pipeline

Get the latest pipeline and reinstall conda environment without mamba.

$ bash scripts/uninstall_conda_env.sh
$ bash scripts/install_conda_env.sh

DO NOT ACTIVATE CONDA ENVIRONMENT before running a pipeline. Pipeline will activate it internally for each task.

hariiyer16 commented 2 years ago

Unless I install the Anaconda and activate the associated conda, I run into issues while doing bash scripts/install_conda_env.sh. Let me try it again. Thanks for hanging in with me.

hariiyer16 commented 2 years ago

Hi Jin, I continued to have issues running the pipeline even after loading java>11. So I did a clean install again - starting with removing caper, Anaconda, atac-seq-pipeline.... Following are the steps I did, which in the end when I instal caper is resulting in a new issue (and I could not find any solution for it online).

Install Anaconda wget https://repo.anaconda.com/archive/Anaconda3-2021.11-Linux-x86_64.sh bash Anaconda3-2021.11-Linux-x86_64.sh -p Anaconda3 -b

Above worked

Activate Anaconda (I have to do this otherwise it uses the system installed Anaconda using which I cannot do conda install of atac-seq-pipeline source Anaconda3/etc/profile.d/conda.sh conda activate base conda info --envs

Above worked without errors

Get atac-seq-pipeline and install the associated conda envs

git clone https://github.com/ENCODE-DCC/atac-seq-pipeline cd ~/atac-seq-pipeline bash scripts/uninstall_conda_env.sh # uninstall it for clean-install bash scripts/install_conda_env.sh # r conda info --envs

Above worked without errors

Install caper

pip install caper conda info --envs

After installing caper, conda gets screwed up and I get the following error even when I call conda info --envs `

ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

Traceback (most recent call last):
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/exceptions.py", line 1080, in __call__
    return func(*args, **kwargs)
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/cli/main.py", line 84, in _main
    exit_code = do_call(args, p)
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/cli/conda_argparse.py", line 81, in do_call
    module = import_module(relative_mod, __name__.rsplit('.', 1)[0])
  File "/users/heaswara/Anaconda3/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/cli/main_info.py", line 19, in <module>
    from ..core.index import _supplement_index_with_system
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/core/index.py", line 13, in <module>
    from .package_cache_data import PackageCacheData
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/core/package_cache_data.py", line 14, in <module>
    from .path_actions import CacheUrlAction, ExtractPackageAction
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/core/path_actions.py", line 30, in <module>
    from ..gateways.connection.download import download
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/gateways/connection/download.py", line 14, in <module>
    from .session import CondaSession
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/gateways/connection/session.py", line 13, in <module>
    from .adapters.s3 import S3Adapter
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/conda/gateways/connection/adapters/s3.py", line 14, in <module>
    boto3.client('s3')  # https://github.com/conda/conda/issues/8993
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/boto3/__init__.py", line 83, in client
    return _get_default_session().client(*args, **kwargs)
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/boto3/session.py", line 258, in client
    return self._session.create_client(
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/session.py", line 815, in create_client
    credentials = self.get_credentials()
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/session.py", line 442, in get_credentials
    self._credentials = self._components.get_component(
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/credentials.py", line 1084, in load_credentials
    creds = provider.load()
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/credentials.py", line 489, in load
    metadata = fetcher.retrieve_iam_role_credentials()
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/utils.py", line 174, in retrieve_iam_role_credentials
    r = self._get_request(url, timeout, num_attempts)
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/utils.py", line 159, in _get_request
    response = requests.get(url, timeout=timeout)
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/vendored/requests/api.py", line 69, in get
    return request('get', url, params=params, **kwargs)
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/vendored/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/vendored/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/vendored/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/vendored/requests/adapters.py", line 360, in send
    resp = conn.urlopen(
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 542, in urlopen
    httplib_response = self._make_request(conn, method, url,
  File "/users/heaswara/Anaconda3/lib/python3.9/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 349, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/users/heaswara/Anaconda3/lib/python3.9/http/client.py", line 1279, in request
    self._send_request(method, url, body, headers, encode_chunked)
TypeError: _send_request() takes 5 positional arguments but 6 were given

`

Any idea what could be happening? I have tried this few times with clean install and keep coming to the same point. Thanks a bunch in advance again.

leepc12 commented 2 years ago

That looks like a conflict between boto3 and Conda? Please don't activate the base Conda environment and pip install caper outside the Conda stuffs and then try again.

hariiyer16 commented 2 years ago

I re-ran without activating base conda. The installation of caper works. But it gets stuck at: 2022-01-12 22:22:57,072|caper.cromwell_workflow_monitor|INFO| Task: id=bca52de2-5ce9-49da-8bb5-4f1553d01ba4, task=atac.read_genome_tsv:-1, retry=0, status=WaitingForReturnCode

Now I remember why I reverted back to fresh Anaconda installation in which case I had to activate this conda.

leepc12 commented 2 years ago

Run qstat to check Caper's job status.

hariiyer16 commented 2 years ago

BTW, I am running the below command when I got the WaitingForReturnCode error: caper run atac.wdl -i https://storage.googleapis.com/encode-pipeline-test-samples/encode-atac-seq-pipeline/ENCSR356KRQ_subsampled.json --conda atac-seq-pipeline

hariiyer16 commented 2 years ago

Run qstat to check Caper's job status.

I already terminated that process .However I was directly running it via command line (not through .sh script invoking the command).

Submitting it again and will get the qstat.

leepc12 commented 2 years ago

Yes, please check both the leader job (that you submitted) and its children jobs (cromwell*) Caper internally calls qsub to submit a shell script for a task (e.g. the read_genome_tsv task). Also, please check your conf file ~/.caper/default.conf. Check sge_resource_param looks good for your cluster.

hariiyer16 commented 2 years ago

Hi Jin, I submitted this as a shell script and the same error is replicated in the error log: 2022-01-15 00:39:14,736|caper.cromwell_workflow_monitor|INFO| Task: id=5411436c-d0c0-4b99-8635-578af7d80092, task=atac.read_genome_tsv:-1, retry=0, status=WaitingForReturnCode

Here is the qstat output: 8821081 0.57800 ATAC-seq_t heaswara r 01/15/2022 00:32:55 shared.q@compute-091.cm.cluste 8

I killed the process eventually.

In the cromwell.out log, I see few lines with WARN, and particularly one line that has: 2022-01-15 00:39:13,003 cromwell-system-akka.dispatchers.engine-dispatcher-40 INFO - 5411436c-d0c0-4b99-8635-578af7d80092-EngineJobExecutionActor-atac.read_genome_tsv:NA:1 [UUID(5411436c)]: Could not copy a suitable cache hit for 5411436c:atac.read_genome_tsv:-1:1. No copy attempts were made.

Please let me know if you need the whole cromwell.out log help.

leepc12 commented 2 years ago

Plesae upload the whole cromwell.out.

hariiyer16 commented 2 years ago

Attaching the cromwell.out and the error log from running the shell script. cromwell.out.txt ATAC-seq_test_error.txt .

leepc12 commented 2 years ago 
    qsub -V -terse -S /bin/bash -N cromwell_5411436c_read_genome_tsv -wd /legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/ATAC_Seq_Pipeline/atac-seq-pipeline/atac/5411436c-d0c0-4b99-8635-578af7d80092/call-read_genome_tsv -o /legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/ATAC_Seq_Pipeline/atac-seq-pipeline/atac/5411436c-d0c0-4b99-8635-578af7d80092/call-read_genome_tsv/execution/stdout -e /legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/ATAC_Seq_Pipeline/atac-seq-pipeline/atac/5411436c-d0c0-4b99-8635-578af7d80092/call-read_genome_tsv/execution/stderr \
         \
          -l h_vmem=$(expr 2048 / 1)m -l s_vmem=$(expr 2048 / 1)m -l h_rt=4:00:00 -l s_rt=4:00:00   \
         \
        /legacy/amber2/scratch/baylin/Hari/Rachael/Ageing_Mouse_Colon_Project/ATAC-Seq/Analyses/OutputDir/ATAC_Seq_Pipeline/atac-seq-pipeline/atac/5411436c-d0c0-4b99-8635-578af7d80092/call-read_genome_tsv/execution/script.caper

Please check if these resource parameters -l h_vmem=$(expr 2048 / 1)m -l s_vmem=$(expr 2048 / 1)m -l h_rt=4:00:00 -l s_rt=4:00:00 work for your cluster's settings.

hariiyer16 commented 2 years ago

Hi Jin, typically we use mem_free and h_vmem parameters, like below; qrsh -l mem_free=2G,h_vmem=2G,h_stack=256M

We don't typically set the s_vmem but doubt that is an issue. In our cluster it is recommended to use h_stack if h_vmem is used. Could that be a problem?

leepc12 commented 2 years ago

I think so, but you should check your job's log first. And then run a test script with something like cat test.sh | qsub -l s_vmem=2G,h_vmem=2G,h_rt=4:00:00,s_rt=4:00:00.

If that's the case then edit caper's conf file ~/.caper/default.conf.

sge-resource-param=${if cpu > 1 then "-pe " + sge_pe + " " else ""} ${if cpu > 1 then cpu else ""} ${true="-l h_vmem=$(expr " false="" defined(memory_mb)}${memory_mb}${true=" / " false="" defined(memory_mb)}${if defined(memory_mb) then cpu else ""}${true=")m" false="" defined(memory_mb)} ${true="-l s_vmem=$(expr " false="" defined(memory_mb)}${memory_mb}${true=" / " false="" defined(memory_mb)}${if defined(memory_mb) then cpu else ""}${true=")m" false="" defined(memory_mb)} ${"-l h_rt=" + time + ":00:00"} ${"-l s_rt=" + time + ":00:00"} ${"-l gpu=" + gpu}

This looks ugly to avoid Cromwell's parsing error. You can simply replace s_vmem with mem_free and append -l h_stack=256M to it.

BTW what about the walltime (h_rt and s_rt)?

hariiyer16 commented 2 years ago

Hi Jin, I haven't been able to get back to this later. Will do and keep you posted. Thank you for the help.