ERROR ~ Error executing process > 'fastcat (1)' with singularity - demo data

[cihaterdogan]

Operating System

Other Linux (please specify below)

Other Linux

Red Hat Enterprise 8.x

Workflow Version

v1.0.3

Workflow Execution

Command line

EPI2ME Version

No response

CLI command run

module load singularity module load java

nextflow is installed to "/path/for/nextflow"

wf-aav-qc data is downloaded to the same folder "/path/for/nextflow"

mkdir -p /path/for/nextflow/singularity

export NXF_SINGULARITY_CACHEDIR=/path/for/nextflow/singularity export SINGULARITY_TMPDIR=/path/for/nextflow/singularity export NXF_HOME=/path/for/nextflow

OUTPUT=wf-aav-qc-demo_results_2

./nextflow run epi2me-labs/wf-aav-qc -r v1.0.3 \ -w ${OUTPUT}/workspace \ -profile singularity \ --threads 8 \ --fastq wf-aav-qc-demo/simulated_reads.fq \ --ref_host wf-aav-qc-demo/cell_line.fasta.gz \ --ref_helper wf-aav-qc-demo/helper.fasta \ --ref_rep_cap wf-aav-qc-demo/repcap.fasta \ --ref_transgene_plasmid wf-aav-qc-demo/transgene.fasta \ --itr1_start 11 \ --itr1_end 156 \ --itr2_start 2156 \ --itr2_end 2286 \ --out_dir ${OUTPUT}

Workflow Execution - CLI Execution Profile

None

What happened?

I'm trying to run the pipeline as a job (with Slurm) on my institution's clusters, but I get the following error.

ERROR ~ Error executing process > 'fastcat (1)'

I was able to successfully run the version v1.0.2 of the workflow with the demo data. I would be grateful for any advice.

Thanks!

Relevant log output

N E X T F L O W  ~  version 23.10.0
Launching `https://github.com/epi2me-labs/wf-aav-qc` [sad_cori] DSL2 - revision: 93ee2a4809 [v1.0.3]

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-aav-qc v1.0.3-g93ee2a4
--------------------------------------------------------------------------------
Core Nextflow options
  revision             : v1.0.3
  runName              : sad_cori
  containerEngine      : singularity
  container            : [withLabel:wf_aav:ontresearch/wf-aav-qc:shad9d16c965b87a03180c6dc5534ad86c96a5d9cb0, withLabel:wf_common:ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9, withLabel:medaka:ontresearch/medaka:sha61a9438d745a78030738352e084445a2db3daa2a]
  launchDir            : /N/scratch/cerdogan/nextflow
  workDir              : /N/scratch/cerdogan/nextflow/wf-aav-qc-demo_results_2/workspace
  projectDir           : /N/scratch/cerdogan/nextflow/assets/epi2me-labs/wf-aav-qc
  userName             : cerdogan
  profile              : singularity
  configFiles          : /N/scratch/cerdogan/nextflow/assets/epi2me-labs/wf-aav-qc/nextflow.config

Input Options
  fastq                : wf-aav-qc-demo/simulated_reads.fq
  itr1_start           : 11
  itr1_end             : 156
  itr2_start           : 2156
  itr2_end             : 2286
  ref_host             : wf-aav-qc-demo/cell_line.fasta.gz
  ref_helper           : wf-aav-qc-demo/helper.fasta
  ref_rep_cap          : wf-aav-qc-demo/repcap.fasta
  ref_transgene_plasmid: wf-aav-qc-demo/transgene.fasta

Output Options
  out_dir              : wf-aav-qc-demo_results_2

Miscellaneous Options
  threads              : 8

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-aav-qc for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-aav-qc v1.0.3-g93ee2a4.
--------------------------------------------------------------------------------
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
[-        ] process > fastcat                -
[-        ] process > pipeline:get_ref_names -

[-        ] process > fastcat                        [  0%] 0 of 1
[-        ] process > pipeline:get_ref_names         [  0%] 0 of 1
[-        ] process > pipeline:medakaVersion         [  0%] 0 of 1
[-        ] process > pipeline:getVersions           -
[-        ] process > pipeline:getParams             [  0%] 0 of 1
[-        ] process > pipeline:mask_transgene_ref... -
[-        ] process > pipeline:make_combined_refe... -
[-        ] process > pipeline:make_index            -
[-        ] process > pipeline:map_to_combined_re... -
[-        ] process > pipeline:truncations           -
[-        ] process > pipeline:itr_coverage          -
[-        ] process > pipeline:contamination         -
[-        ] process > pipeline:aav_structures        -
[-        ] process > pipeline:lookup_medaka_vari... [  0%] 0 of 1
[-        ] process > pipeline:medaka_consensus      -
[-        ] process > pipeline:combine_stats         -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -

executor >  local (5)
[be/7cc051] process > fastcat (1)                    [  0%] 0 of 1
[ba/72169e] process > pipeline:get_ref_names         [  0%] 0 of 1
[d7/6cb84a] process > pipeline:medakaVersion         [  0%] 0 of 1
[-        ] process > pipeline:getVersions           -
[ae/83011b] process > pipeline:getParams             [  0%] 0 of 1
[-        ] process > pipeline:mask_transgene_ref... -
[-        ] process > pipeline:make_combined_refe... -
[-        ] process > pipeline:make_index            -
[-        ] process > pipeline:map_to_combined_re... -
[-        ] process > pipeline:truncations           -
[-        ] process > pipeline:itr_coverage          -
[-        ] process > pipeline:contamination         -
[-        ] process > pipeline:aav_structures        -
[75/b88c3a] process > pipeline:lookup_medaka_vari... [  0%] 0 of 1
[-        ] process > pipeline:medaka_consensus      -
[-        ] process > pipeline:combine_stats         -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -

executor >  local (5)
[be/7cc051] process > fastcat (1)                    [  0%] 0 of 1
[ba/72169e] process > pipeline:get_ref_names         [  0%] 0 of 1
[d7/6cb84a] process > pipeline:medakaVersion         [  0%] 0 of 1
[-        ] process > pipeline:getVersions           -
[ae/83011b] process > pipeline:getParams             [  0%] 0 of 1
[-        ] process > pipeline:mask_transgene_ref... -
[-        ] process > pipeline:make_combined_refe... -
[-        ] process > pipeline:make_index            -
[-        ] process > pipeline:map_to_combined_re... -
[-        ] process > pipeline:truncations           -
[-        ] process > pipeline:itr_coverage          -
[-        ] process > pipeline:contamination         -
[-        ] process > pipeline:aav_structures        -
[75/b88c3a] process > pipeline:lookup_medaka_vari... [  0%] 0 of 1
[-        ] process > pipeline:medaka_consensus      -
[-        ] process > pipeline:combine_stats         -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
ERROR ~ Error executing process > 'fastcat (1)'

Caused by:
  Process `fastcat (1)` terminated with an error exit status (127)

Command executed:

  mkdir fastcat_stats
  fastcat             -s simulated_reads             -r >(bgzip -c > fastcat_stats/per-read-stats.tsv.gz)             -f fastcat_stats/per-file-stats.tsv             --histograms histograms                          input             | bgzip > seqs.fastq.gz

  mv histograms/* fastcat_stats
  # extract the run IDs from the per-read stats
  csvtk cut -tf runid fastcat_stats/per-read-stats.tsv.gz         | csvtk del-header | sort | uniq > fastcat_stats/run_ids

Command exit status:
  127

Command output:
  (empty)

Command error:
  WARNING: passwd file doesn't exist in container, not updating
  WARNING: group file doesn't exist in container, not updating
  .command.sh: line 3: fastcat: command not found
  .command.sh: line 3: bgzip: command not found
  .command.sh: line 3: bgzip: command not found

Work dir:
  /N/scratch/cerdogan/nextflow/wf-aav-qc-demo_results_2/workspace/be/7cc0515224d0f76225c6f3c2ad6d52

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details

executor >  local (5)
[be/7cc051] process > fastcat (1)                    [100%] 1 of 1, failed: 1 ✘
[ba/72169e] process > pipeline:get_ref_names         [  0%] 0 of 1
[d7/6cb84a] process > pipeline:medakaVersion         [  0%] 0 of 1
[-        ] process > pipeline:getVersions           -
[ae/83011b] process > pipeline:getParams             [100%] 1 of 1 ✔
[-        ] process > pipeline:mask_transgene_ref... -
[-        ] process > pipeline:make_combined_refe... -
[-        ] process > pipeline:make_index            -
[-        ] process > pipeline:map_to_combined_re... -
[-        ] process > pipeline:truncations           -
[-        ] process > pipeline:itr_coverage          -
[-        ] process > pipeline:contamination         -
[-        ] process > pipeline:aav_structures        -
[75/b88c3a] process > pipeline:lookup_medaka_vari... [  0%] 0 of 1
[-        ] process > pipeline:medaka_consensus      -
[-        ] process > pipeline:combine_stats         -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
ERROR ~ Error executing process > 'fastcat (1)'

Caused by:
  Process `fastcat (1)` terminated with an error exit status (127)

Command executed:

  mkdir fastcat_stats
  fastcat             -s simulated_reads             -r >(bgzip -c > fastcat_stats/per-read-stats.tsv.gz)             -f fastcat_stats/per-file-stats.tsv             --histograms histograms                          input             | bgzip > seqs.fastq.gz

  mv histograms/* fastcat_stats
  # extract the run IDs from the per-read stats
  csvtk cut -tf runid fastcat_stats/per-read-stats.tsv.gz         | csvtk del-header | sort | uniq > fastcat_stats/run_ids

Command exit status:
  127

Command output:
  (empty)

Command error:
  WARNING: passwd file doesn't exist in container, not updating
  WARNING: group file doesn't exist in container, not updating
  .command.sh: line 3: fastcat: command not found
  .command.sh: line 3: bgzip: command not found
  .command.sh: line 3: bgzip: command not found

Work dir:
  /N/scratch/cerdogan/nextflow/wf-aav-qc-demo_results_2/workspace/be/7cc0515224d0f76225c6f3c2ad6d52

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details
WARN: Killing running tasks (3)

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

other (please describe below)

Other demo data information

I was able to successfully run the version v1.0.2 of the workflow with the demo data.

[nrhorner]

Hi @cihaterdogan

Your command works for me with v1.0.3. Can you confirm that the same command but with -r 1.0.2 does not work for you? This does look to be a SIngularity issue. What version are you using: singularity --version?

[cihaterdogan]

Hi @nrhorner

Yes, the pipeline works with -r v1.0.2 but it doesn't work with -r v1.0.3. I'm using singularity/3.7.2.

Thanks.

[nrhorner]

Hi @cihaterdogan

This is a puzzle to me why you get this error in v1.0.3 but not v1.0.2. Would you be able to paste the output of it 'working' with v1.0.2 please?

[cihaterdogan]

Hi @nrhorner,

Thank you for your reply. The institution deletes unused files in the scratch folder after 30 days. I'm sorry, but I can't share the files you requested.