search

epi2me-labs / wf-clone-validation

Other
25 stars 18 forks source link

Error - failed to polish assembly with Medaka #57

Open r-ovo-hm opened 2 months ago

r-ovo-hm commented 2 months ago

Operating System

macOS

Other Linux

No response

Workflow Version

v1.4.0

Workflow Execution

EPI2ME Desktop (Local)

Other workflow execution

No response

EPI2ME Version

v5.1.14

CLI command run

No response

Workflow Execution - CLI Execution Profile

None

What happened?

Workflow wf-clone-validation stopped with error while executing process 'pipeline:medakaPolishAssembly (1)' From my understanding, it looks like some files are missing from the workflow package (?) I would have expected the workflow to go forward to the production of a alignment report, I am able to localize a fasta file though, although I'm not sure if it's a final assembly result or not

Relevant log output

N E X T F L O W  ~  version 23.04.2
Launching `/Users/minionadmin/epi2melabs/workflows/epi2me-labs/wf-clone-validation/main.nf` [keen_perlman] DSL2 - revision: 3a4d178569

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-clone-validation v1.4.0
--------------------------------------------------------------------------------
Core Nextflow options
  runName        : keen_perlman
  containerEngine: docker
  launchDir      : /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8
  workDir        : /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/work
  projectDir     : /Users/minionadmin/epi2melabs/workflows/epi2me-labs/wf-clone-validation
  userName       : minionadmin
  profile        : standard
  configFiles    : /Users/minionadmin/epi2melabs/workflows/epi2me-labs/wf-clone-validation/nextflow.config

Input Options
  fastq          : /Library/MinKNOW/data/20240830_DNA_QC_run/20240830_DNA_QC/20240830_1204_MN46935_FAZ99665_b8d8576a/fastq_pass/barcode01/FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.fastq

Output Options
  out_dir        : /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/output

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-clone-validation for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x


--------------------------------------------------------------------------------
This is epi2me-labs/wf-clone-validation v1.4.0.
--------------------------------------------------------------------------------
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
[b9/ee7bc9] Submitted process > pipeline:medakaVersion
[3c/c4ff70] Submitted process > fastcat (1)
[4e/4e3177] Submitted process > pipeline:getParams
[b1/e3f76b] Submitted process > pipeline:flyeVersion
[36/38a388] Submitted process > pipeline:getVersions
[87/3f5933] Submitted process > pipeline:checkIfEnoughReads (1)
[40/298405] Submitted process > pipeline:assembleCore (1)
[cf/81ec0b] Submitted process > pipeline:downsampledStats (1)
[15/a98bdc] Submitted process > pipeline:medakaPolishAssembly (1)
ERROR ~ Error executing process > 'pipeline:medakaPolishAssembly (1)'

Caused by:
  Process `pipeline:medakaPolishAssembly (1)` terminated with an error exit status (1)

Command executed:

  STATUS="Failed to polish assembly with Medaka"
  medaka_consensus -i "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.downsampled.fastq" -d "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.reconciled.fasta" -m "dna_r10.4.1_e8.2_400bps_fast@v4.3.0:consensus"         -o . -t 4 -f -q
  echo ">FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11" >> "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.final.fasta"
  sed "2q;d" consensus.fastq >> "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.final.fasta"
  mv consensus.fastq "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.final.fastq"
  STATUS="Completed successfully"

Command exit status:
  1

Command output:
  TF_CPP_MIN_LOG_LEVEL is set to '3'

Command error:
  Cannot import pyabpoa, some features may not be available.
  Cannot import pyabpoa, some features may not be available.
  Failed to interpret 'dna_r10.4.1_e8.2_400bps_fast@v4.3.0:consensus' as a basecaller model.
  Traceback (most recent call last):
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/medaka/medaka.py", line 36, in __call__
      model_fp = medaka.models.resolve_model(val)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/medaka/models.py", line 46, in resolve_model
      raise ValueError(
  ValueError: Model dna_r10.4.1_e8.2_400bps_fast@v4.3.0:consensus is not a known model or existant file.
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "/home/epi2melabs/conda/bin/medaka", line 8, in <module>
      sys.exit(main())
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/medaka/medaka.py", line 832, in main
      args = parser.parse_args()
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1768, in parse_args
      args, argv = self.parse_known_args(args, namespace)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1800, in parse_known_args
      namespace, args = self._parse_known_args(args, namespace)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1988, in _parse_known_args
      positionals_end_index = consume_positionals(start_index)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1965, in consume_positionals
      take_action(action, args)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1874, in take_action
      action(self, namespace, argument_values, option_string)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1159, in __call__
      subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1800, in parse_known_args
      namespace, args = self._parse_known_args(args, namespace)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1988, in _parse_known_args
      positionals_end_index = consume_positionals(start_index)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1965, in consume_positionals
      take_action(action, args)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1874, in take_action
      action(self, namespace, argument_values, option_string)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1159, in __call__
      subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1800, in parse_known_args
      namespace, args = self._parse_known_args(args, namespace)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 2006, in _parse_known_args
      start_index = consume_optional(start_index)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1946, in consume_optional
      take_action(action, args, option_string)
    File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1874, in take_action
      action(self, namespace, argument_values, option_string)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/medaka/medaka.py", line 39, in __call__
      raise RuntimeError(msg.format(self.dest, str(e)))
  RuntimeError: Error validating model from '--model' argument: Model dna_r10.4.1_e8.2_400bps_fast@v4.3.0:consensus is not a known model or existant file..

Work dir:
  /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/work/15/a98bdc9968dbc14b8c2612c7bd9f0c

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '/Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/nextflow.log' file for details


Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

No response

sarahjeeeze commented 2 months ago

Hey, it is hard to say from the info could you share the input cmd and would you mind sharing the nextflow.log file. We have also released a new version v1.5.0 so you could try that. You are right the workflow should still run to completion even though it failed

r-ovo-hm commented 2 months ago

Hi, I have been using the desktop app, so no input command from the user interface. Not sure if this is what you need, but here is the "nextflow intput" that I get from the report: /Applications/EPI2ME.app/Contents/Resources/nextflow-all -log /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/nextflow.log run /Users/minionadmin/epi2melabs/workflows/epi2me-labs/wf-clone-validation/main.nf -params-file /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/params.json -w /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/work -ansi-log false -offline -profile standard -with-weblog 'http://localhost:53529' -resume -name floored_griffiths

(Note that the last part "-resume -name floored_griffiths" corresponds to an attempt to rerun the initial workflow "keen_perlman" that led to the same error)

since then I have run the same workflow, this time using a fasta file as a reference and the worflow failed as well (returning the same error)

From the log files, it looks to my uneducated eye like the medaka model dna_r10.4.1_e8.2_400bps_fast@v4.3.0 is missing (?) I did not get this error when running the demo data.

Please find enclosed the nextflow.log file, I'll try to update the workflow and see if this changes the outcome.

nextflow.log

r-ovo-hm commented 2 months ago

Update: I tried to process the same data with the updated workflow, and this resulted in the same outcome.

cosmohodge commented 2 months ago

Were you able to get this resovled?

r-ovo-hm commented 2 months ago

Hi @cosmohodge, I found a "dirty" workaround that allowed me to proceed with the analysis, but the core issue is not solved. My workaround was to specify wf-clone-validation/test_data/test_medaka_model.tar.gz as the optional --medaka_model_path argument.

cosmohodge commented 2 months ago

Interesting, this workflow was updated 5 days ago in the test_data, updated medaka version.

sarahjeeeze commented 1 month ago

Hi, sorry for this, correct we need to update the models - Will let you know once its done.

sarahjeeeze commented 2 weeks ago

Hi, we are not supporting fast and would encourage the use of HAC or SUP