search

epi2me-labs / wf-bacterial-genomes

Small variant calling for haploid samples
https://labs.epi2me.io/
Other
26 stars 8 forks source link

failing the test_data run #21

Closed splaisan closed 1 year ago

splaisan commented 1 year ago

Operating System

Ubuntu 22.04

Other Linux

No response

Workflow Version

v0.4.0 70714be

Workflow Execution

Command line

EPI2ME Version

No response

CLI command run

running in the repo folder

nextflow run epi2me-labs/wf-bacterial-genomes --fastq ./test_data/fastq --isolates --reference_based_assembly --reference ./test_data/ref/reference.subseq.fa.gz --sample_sheet ./test_data/isolates_sample_sheet.csv --out_dir test_run

Workflow Execution - CLI Execution Profile

standard (default)

What happened?

N E X T F L O W ~ version 23.04.3 NOTE: Your local project version looks outdated - a different revision is available in the remote repository [70714be9ef] Launching https://github.com/epi2me-labs/wf-bacterial-genomes [amazing_carson] DSL2 - revision: d3fdcc0dd3 [master]

ERROR ~ ERROR: Validation of pipeline parameters failed!

-- Check '.nextflow.log' file for details ERROR ~ * --basecaller_cfg: dna_r10.4.1_e8.2_400bps_sup@v4.2.0 is not a valid enum value (dna_r10.4.1_e8.2_400bps_sup@v4.2.0)

-- Check '.nextflow.log' file for details

WARN: Found unexpected parameters:

Relevant log output

Oct-27 14:51:07.248 [main] DEBUG nextflow.cli.Launcher - $> nextflow run epi2me-labs/wf-bacterial-genomes --fastq ./test_data/fastq --isolates --reference_based_assembly --reference ./test_data/ref/reference.subseq.fa.gz --sample_sheet ./test_data/isolates_sample_sheet.csv --out_dir test_run
Oct-27 14:51:07.326 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.04.3
Oct-27 14:51:07.346 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/luna.kuleuven.be/u0002316/.nextflow/plugins; core-plugins: nf-amazon@1.16.2,nf-azure@1.0.1,nf-codecommit@0.1.4,nf-console@1.0.5,nf-ga4gh@1.0.5,nf-google@1.7.3,nf-tower@1.5.12,nf-wave@0.8.4
Oct-27 14:51:07.357 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Oct-27 14:51:07.358 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Oct-27 14:51:07.361 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Oct-27 14:51:07.372 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Oct-27 14:51:07.387 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /home/luna.kuleuven.be/u0002316/.nextflow/scm
Oct-27 14:51:08.272 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/luna.kuleuven.be/u0002316/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-bacterial-genomes.git
Oct-27 14:51:08.301 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
Oct-27 14:51:08.310 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/luna.kuleuven.be/u0002316/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-bacterial-genomes.git
Oct-27 14:51:09.089 [main] INFO  nextflow.scm.AssetManager - NOTE: Your local project version looks outdated - a different revision is available in the remote repository [70714be9ef]
Oct-27 14:51:09.101 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/luna.kuleuven.be/u0002316/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/nextflow.config
Oct-27 14:51:09.101 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: /opt/biotools/wf-bacterial-genomes/nextflow.config
Oct-27 14:51:09.102 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/luna.kuleuven.be/u0002316/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/nextflow.config
Oct-27 14:51:09.102 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /opt/biotools/wf-bacterial-genomes/nextflow.config
Oct-27 14:51:09.112 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Oct-27 14:51:09.246 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Oct-27 14:51:09.452 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Oct-27 14:51:09.454 [main] INFO  nextflow.cli.CmdRun - Launching `https://github.com/epi2me-labs/wf-bacterial-genomes` [amazing_carson] DSL2 - revision: d3fdcc0dd3 [master]
Oct-27 14:51:09.455 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Oct-27 14:51:09.455 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Oct-27 14:51:09.459 [main] DEBUG nextflow.secret.LocalSecretsProvider - Secrets store: /home/luna.kuleuven.be/u0002316/.nextflow/secrets/store.json
Oct-27 14:51:09.461 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@7b18658a] - activable => nextflow.secret.LocalSecretsProvider@7b18658a
Oct-27 14:51:09.519 [main] DEBUG nextflow.Session - Session UUID: 2ae9cbec-7d65-46a6-be31-81f97206f53d
Oct-27 14:51:09.519 [main] DEBUG nextflow.Session - Run name: amazing_carson
Oct-27 14:51:09.519 [main] DEBUG nextflow.Session - Executor pool size: 88
Oct-27 14:51:09.530 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=264; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Oct-27 14:51:09.557 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 23.04.3 build 5875
  Created: 11-08-2023 18:37 UTC (20:37 CEST)
  System: Linux 5.4.0-165-generic
  Runtime: Groovy 3.0.16 on OpenJDK 64-Bit Server VM 17.0.8.1+1-Ubuntu-0ubuntu120.04
  Encoding: UTF-8 (UTF-8)
  Process: 283897@gbw-s-pacbio01 [10.118.132.7]
  CPUs: 88 - Mem: 503.8 GB (2.9 GB) - Swap: 0 (0)
Oct-27 14:51:09.572 [main] DEBUG nextflow.Session - Work-dir: /opt/biotools/wf-bacterial-genomes/work [ext2/ext3]
Oct-27 14:51:09.585 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Oct-27 14:51:09.593 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Oct-27 14:51:09.659 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Oct-27 14:51:09.668 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 89; maxThreads: 1000
Oct-27 14:51:09.724 [main] DEBUG nextflow.Session - Session start
Oct-27 14:51:09.727 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /opt/biotools/wf-bacterial-genomes/test_run/execution/trace.txt
Oct-27 14:51:09.734 [main] DEBUG nextflow.Session - Using default localLib path: /home/luna.kuleuven.be/u0002316/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/lib
Oct-27 14:51:09.736 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/luna.kuleuven.be/u0002316/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/lib
Oct-27 14:51:09.737 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/luna.kuleuven.be/u0002316/.nextflow/assets/epi2me-labs/wf-bacterial-genomes/lib/nfcore_external_java_deps.jar
Oct-27 14:51:11.702 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Oct-27 14:51:12.396 [main] ERROR nextflow.Nextflow - ERROR: Validation of pipeline parameters failed!
Oct-27 14:51:12.408 [main] ERROR nextflow.Nextflow - * --basecaller_cfg: dna_r10.4.1_e8.2_400bps_sup@v4.2.0 is not a valid enum value (dna_r10.4.1_e8.2_400bps_sup@v4.2.0)
Oct-27 14:51:12.412 [main] WARN  nextflow.Nextflow - Found unexpected parameters:
* --mlst_version: 2.23.0
* --flye_opts: null
* --process_label: wfbacterialgenomes
Oct-27 14:51:12.413 [main] INFO  nextflow.Nextflow - - Ignore this warning: params.schema_ignore_params = "mlst_version,flye_opts,process_label"

Application activity log entry

No response

SamStudio8 commented 1 year ago

Hi @splaisan, your log would indicate that you're launching v0.2.14 rather than v0.4.0. Can you try nextflow pull epi2me-labs/wf-bacterial-genomes to update the copy of your workflow and try again? The log should contain a line that says "This is epi2me-labs/wf-bacterial-genomes vX.Y.Z" under our logo to confirm the version number.

NOTE: Your local project version looks outdated - a different revision is available in the remote repository [https://github.com/epi2me-labs/wf-bacterial-genomes/commit/70714be9ef6ebc669b206658471f4a73b907953c]
Launching https://github.com/epi2me-labs/wf-bacterial-genomes [amazing_carson] DSL2 - revision: https://github.com/epi2me-labs/wf-bacterial-genomes/commit/d3fdcc0dd30e8cbc3c992f86e44d843d988f199f [master]
splaisan commented 1 year ago

you are on the right track, I thought I was running the repo code which I just pulled. When I tried to update the nextflow, I got an error

$ nextflow pull epi2me-labs/wf-bacterial-genomes
Checking epi2me-labs/wf-bacterial-genomes ...
Cannot pull project `epi2me-labs/wf-bacterial-genomes` -- org.eclipse.jgit.transport.FetchResult@5c313224
Merge of revisions d3fdcc0dd30e8cbc3c992f86e44d843d988f199f, 70714be9ef6ebc669b206658471f4a73b907953c with base b96bacb5cb6c0c10cc3e9cd63393429f24c7584d using strategy recursive resulted in: Conflicting.

can you please help me fix this? thanks

SamStudio8 commented 1 year ago

@splaisan This typically occurs if the workflow directory has been edited in some way (e.g. to update the nextflow.config file manually). Conflicts can be tricky to resolve correctly, so I would advise to drop and pull the workflow again like so:

nextflow drop epi2me-labs/wf-bacterial-genomes
nextflow pull epi2me-labs/wf-bacterial-genomes
splaisan commented 1 year ago

I'd love to but this also fails

nextflow drop epi2me-labs/wf-bacterial-genomes
Project config file is malformed -- Cause: Compile failed for sources FixedSetSources[name='/groovy/script/ScriptAF8B83CB921349B4D2E0E9DC03D42944/_nf_config_06812e1e']. Cause: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
/groovy/script/ScriptAF8B83CB921349B4D2E0E9DC03D42944/_nf_config_06812e1e: 3: Unexpected input: '{' @ line 3, column 8.
   params {
          ^

1 error

drop worked after deleting my ~/.nextflow folder

splaisan commented 1 year ago

now ruunning but failing at aligning reads after deleting all my nextflow related data in my home folder and pulling the current nextflow, thanks a lot Sam for pointing me to this. I guess the test data is not valid, I will try with my own

SamStudio8 commented 1 year ago

@splaisan No problem, glad you are back on track! Occasionally Nextflow encounters merge errors when pulling workflows. I hadn't appreciated that even drop fails in this case. If you observe this again in future you should be able to overcome this by deleting the specific workflow from your Nextflow asset directory (rather than everything 💥 ), in this case: $HOME/.nextflow/assets/epi2me-labs/wf-bacterial-genomes.

splaisan commented 1 year ago

thanks for the tip, I will btw if you can run my command it means I still have issues. It fails for me when aligning test reads it would be nice to have a test command that works to validate install

SamStudio8 commented 1 year ago

btw if you can run my command it means I still have issues. It fails for me when aligning test reads @splaisan Are you saying the workflow is running but you have a different error? If so, please open a new bug report and we'll take a look 👀

splaisan commented 1 year ago

yes, I repeat the failed command after a new newtflow cleanup and open one now