search

epi2me-labs / wf-bacterial-genomes

Small variant calling for haploid samples
https://labs.epi2me.io/
Other
24 stars 8 forks source link

wf-bacterial-genomes stopped with error #36

Closed liatslu closed 1 month ago

liatslu commented 3 months ago

Operating System

Windows 10

Other Linux

-

Workflow Version

v1.2.0

Workflow Execution

EPI2ME Desktop application

EPI2ME Version

No response

CLI command run

No response

Workflow Execution - CLI Execution Profile

None

What happened?

EPI2ME_issue_export.tar.gz The wf-bacterial-genomes stopped with error. The attached file is generated after clicking "report issue" in the desktop application.

Relevant log output

N E X T F L O W  ~  version 23.04.2
Launching `/mnt/c/Users/lisun/epi2melabs/workflows/epi2me-labs/wf-bacterial-genomes/main.nf` [Cheese-iso-001-genome] DSL2 - revision: 67ff77f941
WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-bacterial-genomes v1.2.0
--------------------------------------------------------------------------------
Core Nextflow options
  runName          : Cheese-iso-001-genome
  containerEngine  : docker
  launchDir        : /mnt/c/Users/lisun/epi2melabs/instances/wf-bacterial-genomes_01J0R9CF7AR99GSCPVW644T2EQ
  workDir          : /mnt/c/Users/lisun/epi2melabs/instances/wf-bacterial-genomes_01J0R9CF7AR99GSCPVW644T2EQ/work
  projectDir       : /mnt/c/Users/lisun/epi2melabs/workflows/epi2me-labs/wf-bacterial-genomes
  userName         : epi2mewsl
  profile          : standard
  configFiles      : /mnt/c/Users/lisun/epi2melabs/workflows/epi2me-labs/wf-bacterial-genomes/nextflow.config, /mnt/c/Users/lisun/epi2melabs/instances/wf-bacterial-genomes_01J0R9CF7AR99GSCPVW644T2EQ/local.config

Input Options
  fastq            : /mnt/g/Cheese-pure-iso/Cheese-iso-001/20240612_2005_MN46760_FAY51710_ba62f739/fastq_pass

Advanced Options
  flye_genome_size : 2800000
  flye_asm_coverage: 50

Output Options
  out_dir          : /mnt/c/Users/lisun/epi2melabs/instances/wf-bacterial-genomes_01J0R9CF7AR99GSCPVW644T2EQ/output

Miscellaneous Options
  threads          : 6

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-bacterial-genomes for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-bacterial-genomes v1.2.0.
--------------------------------------------------------------------------------
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
Running Denovo assembly.
[45/a56415] Submitted process > calling_pipeline:prokkaVersion
[78/5fd76c] Submitted process > calling_pipeline:getParams
[ef/9f32b8] Submitted process > calling_pipeline:lookup_medaka_consensus_model (1)
[ad/e27640] Submitted process > calling_pipeline:lookup_medaka_variant_model (1)
[16/8541a5] Submitted process > fastcat (1)
[57/26ae79] Submitted process > calling_pipeline:medakaVersion
[bd/12b42c] Submitted process > calling_pipeline:mlstVersion
[fe/01ff37] Submitted process > calling_pipeline:getVersions
[13/ef9e13] Submitted process > calling_pipeline:amrCheckpoint (1)
[0c/555ea9] Submitted process > calling_pipeline:ingressCheckpoint (1)
[dc/ba2aed] Submitted process > calling_pipeline:perSampleReportingCheckpoint (1)
[c1/2100a1] Submitted process > calling_pipeline:variantCheckpoint (1)
[32/6ae7a1] Submitted process > calling_pipeline:collectFastqIngressResultsInDir (1)
[cf/f0af7b] Submitted process > calling_pipeline:accumulateCheckpoints (1)
[ed/037feb] Submitted process > calling_pipeline:deNovo (1)
[08/905ab1] Submitted process > calling_pipeline:accumulateCheckpoints (2)
ERROR ~ Error executing process > 'calling_pipeline:deNovo (1)'

Caused by:
  Process `calling_pipeline:deNovo (1)` terminated with an error exit status (1)

Command executed:

  COV_FAIL=0
  FLYE_EXIT_CODE=0
  flye  --genome-size 2800000 --asm-coverage 50 --nano-hq reads.fastq.gz --out-dir output --threads "6" ||     FLYE_EXIT_CODE=$?

  if [[ $FLYE_EXIT_CODE -eq 0 ]]; then
      mv output/assembly.fasta "./fastq_pass.draft_assembly.fasta"
      mv output/assembly_info.txt "./fastq_pass_flye_stats.tsv"
      bgzip "fastq_pass.draft_assembly.fasta"
  else
      # flye failed --> check the log to check why
      edge_cov=$(
          grep -oP 'Mean edge coverage: \K\d+' output/flye.log             || echo 5
      )
      ovlp_cov=$(
          grep -oP 'Overlap-based coverage: \K\d+' output/flye.log             || echo 5
      )
      if [[
          $edge_cov -lt 5 ||
          $ovlp_cov -lt 5
      ]]; then
          echo -n "Caught Flye failure due to low coverage (either mean edge cov. or "
          echo "overlap-based cov. were below 5)".
          COV_FAIL=1
      elif grep -q "No disjointigs were assembled" output/flye.log; then
          echo -n "Caught Flye failure due to disjointig assembly."
          COV_FAIL=2
      else
          # exit a subshell with error so that the process fails
          ( exit $FLYE_EXIT_CODE )
      fi
  fi

Command exit status:
  1

Command output:
  (empty)

Command error:
      ref_seq = aln_reader.get_region_sequence(ctg_region.ctg_id, ctg_region.start,
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/flye/utils/sam_parser.py", line 206, in get_region_sequence
      contig_str = self.ref_fasta[parsed_contig]
    File "<string>", line 2, in __getitem__
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/managers.py", line 834, in _callmethod
      conn.send((self._id, methodname, args, kwds))
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/connection.py", line 206, in send
      self._send_bytes(_ForkingPickler.dumps(obj))
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes
      self._send(header + buf)
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/connection.py", line 368, in _send
      n = write(self._handle, buf)
  BrokenPipeError: [Errno 32] Broken pipe

  Process Process-11:
  Traceback (most recent call last):
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/flye/polishing/bubbles.py", line 71, in _thread_worker
      ref_seq = aln_reader.get_region_sequence(ctg_region.ctg_id, ctg_region.start,
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/flye/utils/sam_parser.py", line 206, in get_region_sequence
      contig_str = self.ref_fasta[parsed_contig]
    File "<string>", line 2, in __getitem__
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/managers.py", line 834, in _callmethod
      conn.send((self._id, methodname, args, kwds))
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/connection.py", line 206, in send
      self._send_bytes(_ForkingPickler.dumps(obj))
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes
      self._send(header + buf)
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/connection.py", line 368, in _send
      n = write(self._handle, buf)
  BrokenPipeError: [Errno 32] Broken pipe

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
      self.run()
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/process.py", line 108, in run
      self._target(*self._args, **self._kwargs)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/flye/polishing/bubbles.py", line 108, in _thread_worker
      error_queue.put(e)
    File "<string>", line 2, in put
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/managers.py", line 834, in _callmethod
      conn.send((self._id, methodname, args, kwds))
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/connection.py", line 206, in send
      self._send_bytes(_ForkingPickler.dumps(obj))
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes
      self._send(header + buf)
    File "/home/epi2melabs/conda/lib/python3.8/multiprocessing/connection.py", line 368, in _send
      n = write(self._handle, buf)
  BrokenPipeError: [Errno 32] Broken pipe

Work dir:
  /mnt/c/Users/lisun/epi2melabs/instances/wf-bacterial-genomes_01J0R9CF7AR99GSCPVW644T2EQ/work/ed/037feb94e83e8046138460d4139d33

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '/mnt/c/Users/lisun/epi2melabs/instances/wf-bacterial-genomes_01J0R9CF7AR99GSCPVW644T2EQ/nextflow.log' file for details
WARN: Killing running tasks (1)

Application activity log entry

No

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

No response

saifdelpiero commented 3 months ago

same problem here :/ did you find any solutions ?

julibeg commented 2 months ago

Hi @liatslu and @saifdelpiero; this could have been caused by the workflow not making enough memory available to Flye. The minimum memory requirement of the workflow (and memory available to Flye) has been increased to 32GB in the most recent release of the workflow (v1.3.0). Could you please update and confirm whether you're seeing the same problem?

Thanks!

cjalder commented 1 month ago

I'm closing this issue for now - please feel free to re-open if updating to the new latest has not resolved this issue.