search

epi2me-labs / wf-alignment

Other
23 stars 15 forks source link

subschemas error #20

Closed kostasgalexiou closed 9 months ago

kostasgalexiou commented 9 months ago

Operating System

CentOS 7

Other Linux

No response

Workflow Version

v0.6.3-gc35c0ec

Workflow Execution

Other (please describe)

EPI2ME Version

No response

CLI command run

sbatch nextflow.sh

nextflow.sh file contents:

#!/bin/bash -x
# # to submit sbatch, sinfo, scancel, squeue
# Le ponemos nombre:
#SBATCH --job-name=npore-align

#Indicamos cuantas tareas y/o equipos necesitamos
# #SBATCH --nodelist=node003
# #SBATCH --partition=p_sequential

#SBATCH --cpus-per-task=2
#SBATCH --mem-per-cpu=16G

# Espacio necesario en /tmp
# # SBATCH --tmp=1000M

module load java

date

~/local/software/nextflow run epi2me-labs/wf-alignment --fastq test.fastq --bam bams --references references/

date
[kalexiou@node027 test]$ ls references/
ERCC.fasta  SIRV_isoforms_multi-fasta_170612a.fasta

Workflow Execution - CLI Execution Profile

standard (default)

What happened?

I am running the test data for the alignment workflow. Process fails with an error regarding number of subschemas (see log output below).

Relevant log output

contents from .nextflow.log file:

nov.-22 09:38:16.790 [main] DEBUG nextflow.cli.Launcher - $> nextflow run epi2me-labs/wf-alignment --fastq test.fastq --bam bams --references references/
nov.-22 09:38:17.039 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 23.10.0
nov.-22 09:38:17.103 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/kalexiou/.nextflow/plugins; core-plugins: nf-amazon@2.1.4,nf-azure@1.3.2,nf-cloudcache@0.3.0,nf-codecommit@0.1.5,nf-console@1.0.6,nf-ga4gh@1.1.0,nf-google@1.8.3,nf-tower@1.6.3,nf-wave@1.0.0
nov.-22 09:38:17.137 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
nov.-22 09:38:17.140 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
nov.-22 09:38:17.148 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
nov.-22 09:38:17.201 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
nov.-22 09:38:17.246 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /home/kalexiou/.nextflow/scm
nov.-22 09:38:19.413 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/kalexiou/.nextflow/assets/epi2me-labs/wf-alignment/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-alignment.git
nov.-22 09:38:19.475 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
nov.-22 09:38:19.506 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/kalexiou/.nextflow/assets/epi2me-labs/wf-alignment/.git/config; branch: master; remote: origin; url: https://github.com/epi2me-labs/wf-alignment.git
nov.-22 09:38:21.460 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/kalexiou/.nextflow/assets/epi2me-labs/wf-alignment/nextflow.config
nov.-22 09:38:21.462 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/kalexiou/.nextflow/assets/epi2me-labs/wf-alignment/nextflow.config
nov.-22 09:38:21.490 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
nov.-22 09:38:21.927 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
nov.-22 09:38:21.928 [main] INFO  nextflow.cli.CmdRun - Launching `https://github.com/epi2me-labs/wf-alignment` [desperate_wright] DSL2 - revision: c35c0ec203 [master]
nov.-22 09:38:21.929 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
nov.-22 09:38:21.929 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
nov.-22 09:38:21.943 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/kalexiou/.nextflow/secrets/store.json
nov.-22 09:38:21.949 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@4a2bf50f] - activable => nextflow.secret.LocalSecretsProvider@4a2bf50f
nov.-22 09:38:22.090 [main] DEBUG nextflow.Session - Session UUID: f3b9f68c-6a4f-4a67-a284-3246c3edd2d8
nov.-22 09:38:22.090 [main] DEBUG nextflow.Session - Run name: desperate_wright
nov.-22 09:38:22.091 [main] DEBUG nextflow.Session - Executor pool size: 2
nov.-22 09:38:22.107 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
nov.-22 09:38:22.128 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=10; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
nov.-22 09:38:22.189 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 23.10.0 build 5889
  Created: 15-10-2023 15:07 UTC (17:07 CEST)
  System: Linux 3.10.0-1062.18.1.el7.x86_64
  Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 11.0.15+10
  Encoding: UTF-8 (UTF-8)
  Process: 25774@node007.bio.crag.local [127.0.1.1]
  CPUs: 2 - Mem: 32 GB (31.7 GB) - Swap: 0 (0)
nov.-22 09:38:22.320 [main] DEBUG nextflow.Session - Work-dir: /scratch/144-cannabis_reseq/Nanopore_JIC_results/test/work [lustre]
nov.-22 09:38:22.389 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
nov.-22 09:38:22.447 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
nov.-22 09:38:22.596 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
nov.-22 09:38:22.619 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 3; maxThreads: 1000
nov.-22 09:38:22.941 [main] DEBUG nextflow.Session - Session start
nov.-22 09:38:22.947 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /scratch/144-cannabis_reseq/Nanopore_JIC_results/test/output/execution/trace.txt
nov.-22 09:38:22.982 [main] DEBUG nextflow.Session - Using default localLib path: /home/kalexiou/.nextflow/assets/epi2me-labs/wf-alignment/lib
nov.-22 09:38:23.054 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/kalexiou/.nextflow/assets/epi2me-labs/wf-alignment/lib
nov.-22 09:38:23.058 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/kalexiou/.nextflow/assets/epi2me-labs/wf-alignment/lib/nfcore_external_java_deps.jar
nov.-22 09:38:33.647 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
nov.-22 09:38:39.781 [main] ERROR nextflow.Nextflow - ERROR: Validation of pipeline parameters failed!
nov.-22 09:38:39.824 [main] ERROR nextflow.Nextflow - * #: 2 subschemas matched instead of one

Application activity log entry

No response

julibeg commented 9 months ago

Hi @kostasgalexiou, you can run the workflow with only either --fastq or --bam as input.

Also, please have a look at the Nextflow documentation for using it with SLURM (https://www.nextflow.io/docs/latest/executor.html#slurm). I'm not very familiar with SLURM, but I think your way of doing it will run Nextflow and all the processes it launches on the same node. If you want to use the cluster properly, you will need to add an extra config to your command (with nextflow run ... -c my-config), which contains process.executor = 'slurm' and cluster-specific options.

Please let us know if you run into any other issues.