search

epi2me-labs / wf-metagenomics

Metagenomic classification of long-read sequencing data
Other
58 stars 23 forks source link

wf-metagenomics (run error) #114

Closed YoungMunLEE closed 1 month ago

YoungMunLEE commented 3 months ago

Operating System

Windows 10

Other Linux

No response

Workflow Version

v2.10.1

Workflow Execution

EPI2ME Desktop (Local)

Other workflow execution

No response

EPI2ME Version

v5.1.14

CLI command run

No response

Workflow Execution - CLI Execution Profile

None

What happened?

As far as I remember from the previous version, the database was automatically downloaded and executed without any additional setup. Has the new version been changed to require manual input database? nextflow.log main.log

This is epi2me-labs/wf-metagenomics v2.10.1.

Checking inputs. ERROR ~ Note: As the classifier parameter is set to minimap2 the database_set parameter must be one of: [ncbi_16s_18s, ncbi_16s_18s_28s_ITS, SILVA_138_1]. Use the kraken2 classifier to access these databases: [Standard-8, PlusPF-8, PlusPFP-8]. -- Check script '/mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics/main.nf' at line: 85 or see '/mnt/c/Users/Youngmun/epi2melabs/instances/wf-metagenomics_01J31NGHT6MGRZJYKGZV63ZT1K/nextflow.log' file for more details

Relevant log output

Jul-18 00:57:46.407 [main] DEBUG nextflow.cli.Launcher - $> /mnt/c/Users/Youngmun/AppData/Local/EPI2ME/app-5.1.14/resources/nextflow-all -log /mnt/c/Users/Youngmun/epi2melabs/instances/wf-metagenomics_01J31NGHT6MGRZJYKGZV63ZT1K/nextflow.log run /mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics/main.nf -params-file /mnt/c/Users/Youngmun/epi2melabs/instances/wf-metagenomics_01J31NGHT6MGRZJYKGZV63ZT1K/params.json -w /mnt/c/Users/Youngmun/epi2melabs/instances/wf-metagenomics_01J31NGHT6MGRZJYKGZV63ZT1K/work -ansi-log false -offline -profile standard -with-weblog 'http://172.25.80.1:58868' -name mystifying_euclid
Jul-18 00:57:46.475 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.04.2
Jul-18 00:57:46.523 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/mnt/c/Users/Youngmun/epi2melabs/workflows/.nextflow/plugins; core-plugins: nf-amazon@1.16.2,nf-azure@1.0.1,nf-codecommit@0.1.4,nf-console@1.0.5,nf-ga4gh@1.0.5,nf-google@1.7.3,nf-tower@1.5.12,nf-wave@0.8.3
Jul-18 00:57:46.547 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jul-18 00:57:46.550 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jul-18 00:57:46.557 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jul-18 00:57:46.582 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Jul-18 00:57:46.611 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics/nextflow.config
Jul-18 00:57:46.614 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics/nextflow.config
Jul-18 00:57:46.668 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Jul-18 00:57:48.021 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Jul-18 00:57:48.037 [main] INFO  nextflow.cli.CmdRun - Launching `/mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics/main.nf` [mystifying_euclid] DSL2 - revision: 6247543ce2
Jul-18 00:57:48.038 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Jul-18 00:57:48.038 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Jul-18 00:57:48.073 [main] DEBUG nextflow.secret.LocalSecretsProvider - Secrets store: /mnt/c/Users/Youngmun/epi2melabs/workflows/.nextflow/secrets/store.json
Jul-18 00:57:48.079 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@19fd43da] - activable => nextflow.secret.LocalSecretsProvider@19fd43da
Jul-18 00:57:48.205 [main] DEBUG nextflow.Session - Session UUID: 7f1a458c-3e02-4a92-86f1-b9d6dc520cd5
Jul-18 00:57:48.205 [main] DEBUG nextflow.Session - Run name: mystifying_euclid
Jul-18 00:57:48.206 [main] DEBUG nextflow.Session - Executor pool size: 8
Jul-18 00:57:48.227 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=24; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jul-18 00:57:48.254 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 23.04.2 build 5871
  Created: 08-06-2023 09:33 UTC (09:33 GMT)
  System: Linux 5.15.153.1-microsoft-standard-WSL2
  Runtime: Groovy 3.0.16 on OpenJDK 64-Bit Server VM 11.0.21+9-post-Ubuntu-0ubuntu122.04
  Encoding: UTF-8 (UTF-8)
  Process: 25183@DESKTOP-8VN12HA [127.0.1.1]
  CPUs: 8 - Mem: 11.7 GB (10.3 GB) - Swap: 3 GB (3 GB)
Jul-18 00:57:48.299 [main] DEBUG nextflow.Session - Work-dir: /mnt/c/Users/Youngmun/epi2melabs/instances/wf-metagenomics_01J31NGHT6MGRZJYKGZV63ZT1K/work [v9fs]
Jul-18 00:57:48.370 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[GoogleLifeSciencesExecutor, AwsBatchExecutor, GoogleBatchExecutor]
Jul-18 00:57:48.392 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Jul-18 00:57:48.447 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 9; maxThreads: 1000
Jul-18 00:57:48.461 [main] DEBUG nextflow.Session - Observer factory: TowerFactory
Jul-18 00:57:48.542 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Jul-18 00:57:48.709 [main] DEBUG nextflow.Session - Session start
Jul-18 00:57:48.716 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /mnt/c/Users/Youngmun/epi2melabs/instances/wf-metagenomics_01J31NGHT6MGRZJYKGZV63ZT1K/output/execution/trace.txt
Jul-18 00:57:48.759 [main] DEBUG nextflow.Session - Using default localLib path: /mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics/lib
Jul-18 00:57:48.795 [main] DEBUG nextflow.Session - Adding to the classpath library: /mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics/lib
Jul-18 00:57:48.798 [main] DEBUG nextflow.Session - Adding to the classpath library: /mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics/lib/nfcore_external_java_deps.jar
Jul-18 00:57:48.966 [Actor Thread 2] DEBUG nextflow.trace.WebLogObserver - Successfully sent message to http://172.25.80.1:58868 -- received status code 200.
Jul-18 00:57:50.905 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Jul-18 00:57:57.965 [main] WARN  nextflow.NextflowMeta$Preview - NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
Jul-18 00:57:58.223 [main] INFO  nextflow.Nextflow - 
||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-metagenomics v2.10.1
--------------------------------------------------------------------------------
Core Nextflow options
  runName        : mystifying_euclid
  containerEngine: docker
  launchDir      : /mnt/c/Users/Youngmun/epi2melabs/instances/wf-metagenomics_01J31NGHT6MGRZJYKGZV63ZT1K
  workDir        : /mnt/c/Users/Youngmun/epi2melabs/instances/wf-metagenomics_01J31NGHT6MGRZJYKGZV63ZT1K/work
  projectDir     : /mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics
  userName       : epi2mewsl
  profile        : standard
  configFiles    : /mnt/c/Users/Youngmun/epi2melabs/workflows/epi2me-labs/wf-metagenomics/nextflow.config

Input Options
  fastq          : /mnt/c/Users/Youngmun/Downloads/20240702_1412_MN39891_AUA801_d83d9b84/pod5/basecalling/pass
  classifier     : minimap2

Reference Options
  store_dir      : /mnt/c/Users/Youngmun/epi2melabs/data
  database_sets  : [ncbi_16s_18s:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_16s_18s.fna, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_kraken2.tar.gz, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ref2taxid.targloci.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], ncbi_16s_18s_28s_ITS:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS.fna, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS_kraken2.tar.gz, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ref2taxid.ncbi_16s_18s_28s_ITS.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], SILVA_138_1:[database:null], Standard-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_standard_08gb_20231009.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip], PlusPF-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_pluspf_08gb_20230314.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip], PlusPFP-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_pluspfp_08gb_20230314.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip]]

Output Options
  out_dir        : /mnt/c/Users/Youngmun/epi2melabs/instances/wf-metagenomics_01J31NGHT6MGRZJYKGZV63ZT1K/output

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-metagenomics for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-metagenomics v2.10.1.
--------------------------------------------------------------------------------
Jul-18 00:57:59.580 [main] INFO  nextflow.Nextflow - Checking inputs.
Jul-18 00:57:59.586 [main] DEBUG nextflow.Session - Session aborted -- Cause: Note: As the classifier parameter is set to `minimap2` the `database_set` parameter must be one of: [ncbi_16s_18s, ncbi_16s_18s_28s_ITS, SILVA_138_1]. Use the `kraken2` classifier to access these databases: [Standard-8, PlusPF-8, PlusPFP-8].
Jul-18 00:57:59.603 [Actor Thread 3] DEBUG nextflow.trace.WebLogObserver - Successfully sent message to http://172.25.80.1:58868 -- received status code 200.
Jul-18 00:57:59.885 [main] ERROR nextflow.cli.Launcher - @unknown
java.lang.Exception: Note: As the classifier parameter is set to `minimap2` the `database_set` parameter must be one of: [ncbi_16s_18s, ncbi_16s_18s_28s_ITS, SILVA_138_1]. Use the `kraken2` classifier to access these databases: [Standard-8, PlusPF-8, PlusPFP-8].
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:72)
    at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:59)
    at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:59)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:263)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:277)
    at Script_f35efa24$_runScript_closure1$_closure4.doCall(Script_f35efa24:85)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
    at groovy.lang.Closure.call(Closure.java:412)
    at groovy.lang.Closure.call(Closure.java:406)
    at nextflow.script.WorkflowDef.run0(WorkflowDef.groovy:204)
    at nextflow.script.WorkflowDef.run(WorkflowDef.groovy:188)
    at nextflow.script.BindableDef.invoke_a(BindableDef.groovy:51)
    at nextflow.script.IterableDef$invoke_a.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
    at nextflow.script.BaseScript.run0(BaseScript.groovy:183)
    at nextflow.script.BaseScript.run(BaseScript.groovy:192)
    at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:229)
    at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:224)
    at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:130)
    at nextflow.cli.CmdRun.run(CmdRun.groovy:368)
    at nextflow.cli.Launcher.run(Launcher.groovy:494)
    at nextflow.cli.Launcher.main(Launcher.groovy:653)

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

No response

nggvs commented 3 months ago

Hi @YoungMunLEE , The wf includes default databases that are downloaded and stored. The default database is the Standard-8 which is available for the kraken2 pipeline. If you wish to use minimap2, you'd need to choose one of the next: ncbi_16s_18s, ncbi_16s_18s_28s_ITS, SILVA_138_1 or provide your own one. Thank you very much for using the workflow!

YoungMunLEE commented 3 months ago

Understood, and thank you for your response. I have another question. When using MinKNOW software for sequencing and basecalling, some people enable the barcode trim option while others do not. When uploading fastq files to wf-metagenomics, is it acceptable to upload fastq files that have already been trimmed for barcodes? My question is whether the results will differ depending on whether the barcodes have been trimmed. Thank you inadvance.

nggvs commented 1 month ago

Hi @YoungMunLEE , You can upload fastq whose barcodes have been trimmed as input data. The workflow doesn't remove them per se. Thank you!

nggvs commented 1 month ago

I'm going to close this issue as the first question has been already answered, but please feel free to open new ones if more doubts appear. It helps also to other people as they can check the title of the issue to read about it. Thank you very much for using the workflow!