Error - failed to polish assembly with Medaka

r-ovo-hm commented 2 weeks ago

Operating System

macOS

Other Linux

No response

Workflow Version

v1.4.0

Workflow Execution

EPI2ME Desktop (Local)

Other workflow execution

No response

EPI2ME Version

v5.1.14

CLI command run

No response

Workflow Execution - CLI Execution Profile

None

What happened?

Workflow wf-clone-validation stopped with error while executing process 'pipeline:medakaPolishAssembly (1)' From my understanding, it looks like some files are missing from the workflow package (?) I would have expected the workflow to go forward to the production of a alignment report, I am able to localize a fasta file though, although I'm not sure if it's a final assembly result or not

Relevant log output

N E X T F L O W  ~  version 23.04.2 Launching `/Users/minionadmin/epi2melabs/workflows/epi2me-labs/wf-clone-validation/main.nf` [keen_perlman] DSL2 - revision: 3a4d178569  ||||||||||   _____ ____ ___ ____  __  __ _____      _       _ ||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___ |||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __| |||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \ ||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/ ||||||||||  wf-clone-validation v1.4.0 -------------------------------------------------------------------------------- Core Nextflow options   runName        : keen_perlman   containerEngine: docker   launchDir      : /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8   workDir        : /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/work   projectDir     : /Users/minionadmin/epi2melabs/workflows/epi2me-labs/wf-clone-validation   userName       : minionadmin   profile        : standard   configFiles    : /Users/minionadmin/epi2melabs/workflows/epi2me-labs/wf-clone-validation/nextflow.config  Input Options   fastq          : /Library/MinKNOW/data/20240830_DNA_QC_run/20240830_DNA_QC/20240830_1204_MN46935_FAZ99665_b8d8576a/fastq_pass/barcode01/FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.fastq  Output Options   out_dir        : /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/output  !! Only displaying parameters that differ from the pipeline defaults !! -------------------------------------------------------------------------------- If you use epi2me-labs/wf-clone-validation for your analysis please cite:  * The nf-core framework   https://doi.org/10.1038/s41587-020-0439-x   -------------------------------------------------------------------------------- This is epi2me-labs/wf-clone-validation v1.4.0. -------------------------------------------------------------------------------- Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files. [b9/ee7bc9] Submitted process > pipeline:medakaVersion [3c/c4ff70] Submitted process > fastcat (1) [4e/4e3177] Submitted process > pipeline:getParams [b1/e3f76b] Submitted process > pipeline:flyeVersion [36/38a388] Submitted process > pipeline:getVersions [87/3f5933] Submitted process > pipeline:checkIfEnoughReads (1) [40/298405] Submitted process > pipeline:assembleCore (1) [cf/81ec0b] Submitted process > pipeline:downsampledStats (1) [15/a98bdc] Submitted process > pipeline:medakaPolishAssembly (1) ERROR ~ Error executing process > 'pipeline:medakaPolishAssembly (1)'  Caused by:   Process `pipeline:medakaPolishAssembly (1)` terminated with an error exit status (1)  Command executed:    STATUS="Failed to polish assembly with Medaka"   medaka_consensus -i "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.downsampled.fastq" -d "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.reconciled.fasta" -m "dna_r10.4.1_e8.2_400bps_fast@v4.3.0:consensus"         -o . -t 4 -f -q   echo ">FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11" >> "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.final.fasta"   sed "2q;d" consensus.fastq >> "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.final.fasta"   mv consensus.fastq "FAZ99665_pass_barcode01_b8d8576a_6046d4f4_11.final.fastq"   STATUS="Completed successfully"  Command exit status:   1  Command output:   TF_CPP_MIN_LOG_LEVEL is set to '3'  Command error:   Cannot import pyabpoa, some features may not be available.   Cannot import pyabpoa, some features may not be available.   Failed to interpret 'dna_r10.4.1_e8.2_400bps_fast@v4.3.0:consensus' as a basecaller model.   Traceback (most recent call last):     File "/home/epi2melabs/conda/lib/python3.8/site-packages/medaka/medaka.py", line 36, in __call__       model_fp = medaka.models.resolve_model(val)     File "/home/epi2melabs/conda/lib/python3.8/site-packages/medaka/models.py", line 46, in resolve_model       raise ValueError(   ValueError: Model dna_r10.4.1_e8.2_400bps_fast@v4.3.0:consensus is not a known model or existant file.      During handling of the above exception, another exception occurred:      Traceback (most recent call last):     File "/home/epi2melabs/conda/bin/medaka", line 8, in <module>       sys.exit(main())     File "/home/epi2melabs/conda/lib/python3.8/site-packages/medaka/medaka.py", line 832, in main       args = parser.parse_args()     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1768, in parse_args       args, argv = self.parse_known_args(args, namespace)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1800, in parse_known_args       namespace, args = self._parse_known_args(args, namespace)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1988, in _parse_known_args       positionals_end_index = consume_positionals(start_index)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1965, in consume_positionals       take_action(action, args)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1874, in take_action       action(self, namespace, argument_values, option_string)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1159, in __call__       subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1800, in parse_known_args       namespace, args = self._parse_known_args(args, namespace)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1988, in _parse_known_args       positionals_end_index = consume_positionals(start_index)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1965, in consume_positionals       take_action(action, args)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1874, in take_action       action(self, namespace, argument_values, option_string)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1159, in __call__       subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1800, in parse_known_args       namespace, args = self._parse_known_args(args, namespace)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 2006, in _parse_known_args       start_index = consume_optional(start_index)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1946, in consume_optional       take_action(action, args, option_string)     File "/home/epi2melabs/conda/lib/python3.8/argparse.py", line 1874, in take_action       action(self, namespace, argument_values, option_string)     File "/home/epi2melabs/conda/lib/python3.8/site-packages/medaka/medaka.py", line 39, in __call__       raise RuntimeError(msg.format(self.dest, str(e)))   RuntimeError: Error validating model from '--model' argument: Model dna_r10.4.1_e8.2_400bps_fast@v4.3.0:consensus is not a known model or existant file..  Work dir:   /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/work/15/a98bdc9968dbc14b8c2612c7bd9f0c  Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`   -- Check '/Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/nextflow.log' file for details

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

No response

sarahjeeeze commented 1 week ago

Hey, it is hard to say from the info could you share the input cmd and would you mind sharing the nextflow.log file. We have also released a new version v1.5.0 so you could try that. You are right the workflow should still run to completion even though it failed

r-ovo-hm commented 1 week ago

Hi, I have been using the desktop app, so no input command from the user interface. Not sure if this is what you need, but here is the "nextflow intput" that I get from the report: /Applications/EPI2ME.app/Contents/Resources/nextflow-all -log /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/nextflow.log run /Users/minionadmin/epi2melabs/workflows/epi2me-labs/wf-clone-validation/main.nf -params-file /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/params.json -w /Users/minionadmin/epi2melabs/instances/wf-clone-validation_01J7E6YDSN2B00VRPQP54NC3Z8/work -ansi-log false -offline -profile standard -with-weblog 'http://localhost:53529' -resume -name floored_griffiths

(Note that the last part "-resume -name floored_griffiths" corresponds to an attempt to rerun the initial workflow "keen_perlman" that led to the same error)

since then I have run the same workflow, this time using a fasta file as a reference and the worflow failed as well (returning the same error)

From the log files, it looks to my uneducated eye like the medaka model dna_r10.4.1_e8.2_400bps_fast@v4.3.0 is missing (?) I did not get this error when running the demo data.

Please find enclosed the nextflow.log file, I'll try to update the workflow and see if this changes the outcome.

nextflow.log

r-ovo-hm commented 1 week ago

Update: I tried to process the same data with the updated workflow, and this resulted in the same outcome.

cosmohodge commented 6 days ago

Were you able to get this resovled?

r-ovo-hm commented 6 days ago

Hi @cosmohodge, I found a "dirty" workaround that allowed me to proceed with the analysis, but the core issue is not solved. My workaround was to specify wf-clone-validation/test_data/test_medaka_model.tar.gz as the optional --medaka_model_path argument.

cosmohodge commented 6 days ago

Interesting, this workflow was updated 5 days ago in the test_data, updated medaka version.

epi2me-labs / wf-clone-validation