search

nf-core / mag

Assembly and binning of metagenomes
https://nf-co.re/mag
MIT License
206 stars 104 forks source link

ERROR nextflow.cli.Launcher - Is a directory #478

Closed Pandapip1 closed 1 year ago

Pandapip1 commented 1 year ago

Description of the bug

When I attempt to run nf-core/mag, the script, immediately after startup, errors with the following unusual output:

Is a directory

 -- Check script '/ref/djslab/data/nextflow/nf-core/mag/2_3_2/./workflows/../subworkflows/local/gunc_qc.nf' at line: 7 or see '.nextflow.log' file for more details

Command used and terminal output

gavinjohn@login:/scratch/djslab/gavinjohn $ nextflow run /ref/djslab/data/nextflow/nf-core/mag/2_3_2 -profile test --outdir test
N E X T F L O W  ~  version 22.04.3
Launching `/ref/djslab/data/nextflow/nf-core/mag/2_3_2/main.nf` [marvelous_banach] DSL2 - revision: 8f9adbafb4

------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/mag v2.3.2
------------------------------------------------------
Core Nextflow options
  runName                    : marvelous_banach
  containerEngine            : singularity
  launchDir                  : /scratch/djslab/gavinjohn
  workDir                    : /scratch/djslab/gavinjohn/work
  projectDir                 : /ref/djslab/data/nextflow/nf-core/mag/2_3_2
  userName                   : gavinjohn
  profile                    : test
  configFiles                : /home/gavinjohn/.nextflow/config, /ref/djslab/data/nextflow/nf-core/mag/2_3_2/nextflow.config

Input/output options
  input                      : https://raw.githubusercontent.com/nf-core/test-datasets/mag/samplesheets/samplesheet.csv
  outdir                     : test

Institutional config options
  custom_config_base         : /ref/djslab/data/nextflow/nf-core/mag/2_3_2/../configs/
  config_profile_name        : Test profile
  config_profile_description : Minimal test dataset to check pipeline function

Max job request options
  max_cpus                   : 2
  max_memory                 : 6.GB
  max_time                   : 6.h

Quality control for short reads options
  phix_reference             : /ref/djslab/data/nextflow/nf-core/mag/2_3_2/assets/data/GCA_002596845.1_ASM259684v1_genomic.fna.gz

Quality control for long reads options
  lambda_reference           : /ref/djslab/data/nextflow/nf-core/mag/2_3_2/assets/data/GCA_000840245.1_ViralProj14204_genomic.fna.gz

Taxonomic profiling options
  centrifuge_db              : https://raw.githubusercontent.com/nf-core/test-datasets/mag/test_data/minigut_cf.tar.gz
  kraken2_db                 : https://raw.githubusercontent.com/nf-core/test-datasets/mag/test_data/minigut_kraken.tgz
  skip_krona                 : true
  gtdb                       : false

Binning options
  skip_concoct               : true
  min_length_unbinned_contigs: 1
  max_unbinned_contigs       : 2

Bin quality check options
  busco_reference            : https://busco-data.ezlab.org/v5/data/lineages/bacteria_odb10.2020-03-06.tar.gz
  busco_clean                : true
  checkm_db                  : null
  gunc_db                    : null

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/mag for your analysis please cite:

* The pipeline publication
  https://doi.org/10.1093/nargab/lqac007

* The pipeline
  https://doi.org/10.5281/zenodo.3589527

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/mag/blob/master/CITATIONS.md
------------------------------------------------------
Is a directory

 -- Check script '/ref/djslab/data/nextflow/nf-core/mag/2_3_2/./workflows/../subworkflows/local/gunc_qc.nf' at line: 7 or see '.nextflow.log' file for more details
gavinjohn@login:/scratch/djslab/gavinjohn $ nextfnextflow run /ref/djslab/data/nextflow/nf-core/mag/2_3_2 --input /scratch/djslab/gavinjohn/lts/djslab/DJSLAB/projects/MollyDrewSRAreads/SRR3131692_FW_CLEAN.fastq.gz --outdir test
N E X T F L O W  ~  version 22.04.3
Launching `/ref/djslab/data/nextflow/nf-core/mag/2_3_2/main.nf` [reverent_hypatia] DSL2 - revision: 8f9adbafb4

------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/mag v2.3.2
------------------------------------------------------
Core Nextflow options
  runName           : reverent_hypatia
  containerEngine   : singularity
  launchDir         : /scratch/djslab/gavinjohn
  workDir           : /scratch/djslab/gavinjohn/work
  projectDir        : /ref/djslab/data/nextflow/nf-core/mag/2_3_2
  userName          : gavinjohn
  profile           : standard
  configFiles       : /home/gavinjohn/.nextflow/config, /ref/djslab/data/nextflow/nf-core/mag/2_3_2/nextflow.config

Input/output options
  input             : /scratch/djslab/gavinjohn/lts/djslab/DJSLAB/projects/MollyDrewSRAreads/SRR3131692_FW_CLEAN.fastq.gz
  outdir            : test

Institutional config options
  custom_config_base: /ref/djslab/data/nextflow/nf-core/mag/2_3_2/../configs/

Quality control for short reads options
  phix_reference    : /ref/djslab/data/nextflow/nf-core/mag/2_3_2/assets/data/GCA_002596845.1_ASM259684v1_genomic.fna.gz

Quality control for long reads options
  lambda_reference  : /ref/djslab/data/nextflow/nf-core/mag/2_3_2/assets/data/GCA_000840245.1_ViralProj14204_genomic.fna.gz

Taxonomic profiling options
  gtdb              : https://data.ace.uq.edu.au/public/gtdb/data/releases/release202/202.0/auxillary_files/gtdbtk_r202_data.tar.gz

Bin quality check options
  checkm_db         : null
  gunc_db           : null

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/mag for your analysis please cite:

* The pipeline publication
  https://doi.org/10.1093/nargab/lqac007

* The pipeline
  https://doi.org/10.5281/zenodo.3589527

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/mag/blob/master/CITATIONS.md
------------------------------------------------------
Is a directory

 -- Check script '/ref/djslab/data/nextflow/nf-core/mag/2_3_2/./workflows/../subworkflows/local/gunc_qc.nf' at line: 7 or see '.nextflow.log' file for more details

Output is identical in both cases.

Relevant files

.nextflow.log

Jul-17 11:53:59.337 [main] DEBUG nextflow.cli.Launcher - $> nextflow run /ref/djslab/data/nextflow/nf-core/mag/2_3_2 -profile test --outdir test
Jul-17 11:54:01.617 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 22.04.3
Jul-17 11:54:03.223 [main] DEBUG nextflow.config.ConfigBuilder - Found config home: /home/gavinjohn/.nextflow/config
Jul-17 11:54:03.228 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /ref/djslab/data/nextflow/nf-core/mag/2_3_2/nextflow.config
Jul-17 11:54:03.242 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/gavinjohn/.nextflow/config
Jul-17 11:54:03.246 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /ref/djslab/data/nextflow/nf-core/mag/2_3_2/nextflow.config
Jul-17 11:54:03.264 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `test`
Jul-17 11:54:03.419 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `test`
Jul-17 11:54:06.055 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [cfc_dev, ifb_core, denbi_qbic, alice, mjolnir_globe, uppmax, incliva, uge, rosalind_uge, lugh, unibe_ibu, vai, czbiohub_aws, jax, ccga_med, scw, tigem, tubingen_apg, google, ipop_up, googlels, eddie, medair, test_host_rm, bi, bigpurple, sbc_sharc, adcra, cedars, vsc_kul_uhasselt, pawsey_nimbus, ucl_myriad, utd_ganymede, charliecloud, icr_davros, test_ancient_dna, ceres, munin, arm, rosalind, test_binrefinement, hasta, cfc, uzh, test_hybrid_host_rm, test_busco_auto, ebc, ku_sund_dangpu, ccga_dx, test_hybrid, crick, marvin, biohpc_gen, shifter, mana, mamba, wehi, awsbatch, imperial, maestro, genotoul, test_no_clipping, abims, janelia, nihbiowulf, nu_genomics, googlebatch, oist, sahmri, mpcdf, leicester, vsc_ugent, sage, cambridge, podman, cheaha, xanadu, test, computerome, seg_globe, sanger, dkfz, pasteur, test_full, azurebatch, hki, test_adapterremoval, crukmi, docker, engaging, gis, hypatia, psmn, eva, fgcz, conda, crg, singularity, uw_hyak_pedslabs, prince, utd_sysbio, debug, genouest, cbe, phoenix, gitpod, seawulf, uct_hpc, aws_tower, binac]
Jul-17 11:54:06.152 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Jul-17 11:54:06.643 [main] INFO  nextflow.cli.CmdRun - Launching `/ref/djslab/data/nextflow/nf-core/mag/2_3_2/main.nf` [marvelous_banach] DSL2 - revision: 8f9adbafb4
Jul-17 11:54:06.814 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/home/gavinjohn/.nextflow/plugins; core-plugins: nf-amazon@1.7.2,nf-azure@0.13.2,nf-console@1.0.3,nf-ga4gh@1.0.3,nf-google@1.1.4,nf-sqldb@0.4.0,nf-tower@1.4.0
Jul-17 11:54:06.824 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Jul-17 11:54:06.861 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jul-17 11:54:06.868 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jul-17 11:54:06.883 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jul-17 11:54:06.907 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Jul-17 11:54:07.014 [main] DEBUG nextflow.Session - Session uuid: c30527e9-1c66-4f19-bfac-7116050217d9
Jul-17 11:54:07.049 [main] DEBUG nextflow.Session - Run name: marvelous_banach
Jul-17 11:54:07.058 [main] DEBUG nextflow.Session - Executor pool size: 48
Jul-17 11:54:07.133 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 22.04.3 build 5703
  Created: 18-05-2022 19:22 UTC (14:22 CDT)
  System: Linux 5.4.233-1.el8.elrepo.x86_64
  Runtime: Groovy 3.0.10 on OpenJDK 64-Bit Server VM 11.0.15+10
  Encoding: UTF-8 (UTF-8)
  Process: 3225032@login [192.168.80.32]
  CPUs: 48 - Mem: 187.6 GB (5.8 GB) - Swap: 3.9 GB (817.5 MB)
Jul-17 11:54:07.206 [main] DEBUG nextflow.Session - Work-dir: /scratch/djslab/gavinjohn/work [fhgfs]
Jul-17 11:54:07.235 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Jul-17 11:54:07.262 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Jul-17 11:54:07.498 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Jul-17 11:54:07.528 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 49; maxThreads: 1000
Jul-17 11:54:11.635 [main] DEBUG nextflow.Session - Session start invoked
Jul-17 11:54:11.655 [main] DEBUG nextflow.trace.TraceFileObserver - Flow starting -- trace file: /scratch/djslab/gavinjohn/test/pipeline_info/execution_trace_2023-07-17_11-54-05.txt
Jul-17 11:54:11.690 [main] DEBUG nextflow.Session - Using default localLib path: /ref/djslab/data/nextflow/nf-core/mag/2_3_2/lib
Jul-17 11:54:11.703 [main] DEBUG nextflow.Session - Adding to the classpath library: /ref/djslab/data/nextflow/nf-core/mag/2_3_2/lib
Jul-17 11:54:11.706 [main] DEBUG nextflow.Session - Adding to the classpath library: /ref/djslab/data/nextflow/nf-core/mag/2_3_2/lib/nfcore_external_java_deps.jar
Jul-17 11:54:13.272 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Jul-17 11:54:13.909 [main] INFO  nextflow.Nextflow - 

------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/mag v2.3.2
------------------------------------------------------
Core Nextflow options
  runName                    : marvelous_banach
  containerEngine            : singularity
  launchDir                  : /scratch/djslab/gavinjohn
  workDir                    : /scratch/djslab/gavinjohn/work
  projectDir                 : /ref/djslab/data/nextflow/nf-core/mag/2_3_2
  userName                   : gavinjohn
  profile                    : test
  configFiles                : /home/gavinjohn/.nextflow/config, /ref/djslab/data/nextflow/nf-core/mag/2_3_2/nextflow.config

Input/output options
  input                      : https://raw.githubusercontent.com/nf-core/test-datasets/mag/samplesheets/samplesheet.csv
  outdir                     : test

Institutional config options
  custom_config_base         : /ref/djslab/data/nextflow/nf-core/mag/2_3_2/../configs/
  config_profile_name        : Test profile
  config_profile_description : Minimal test dataset to check pipeline function

Max job request options
  max_cpus                   : 2
  max_memory                 : 6.GB
  max_time                   : 6.h

Quality control for short reads options
  phix_reference             : /ref/djslab/data/nextflow/nf-core/mag/2_3_2/assets/data/GCA_002596845.1_ASM259684v1_genomic.fna.gz

Quality control for long reads options
  lambda_reference           : /ref/djslab/data/nextflow/nf-core/mag/2_3_2/assets/data/GCA_000840245.1_ViralProj14204_genomic.fna.gz

Taxonomic profiling options
  centrifuge_db              : https://raw.githubusercontent.com/nf-core/test-datasets/mag/test_data/minigut_cf.tar.gz
  kraken2_db                 : https://raw.githubusercontent.com/nf-core/test-datasets/mag/test_data/minigut_kraken.tgz
  skip_krona                 : true
  gtdb                       : false

Binning options
  skip_concoct               : true
  min_length_unbinned_contigs: 1
  max_unbinned_contigs       : 2

Bin quality check options
  busco_reference            : https://busco-data.ezlab.org/v5/data/lineages/bacteria_odb10.2020-03-06.tar.gz
  busco_clean                : true
  checkm_db                  : null
  gunc_db                    : null

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/mag for your analysis please cite:

* The pipeline publication
  https://doi.org/10.1093/nargab/lqac007

* The pipeline
  https://doi.org/10.5281/zenodo.3589527

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/mag/blob/master/CITATIONS.md
------------------------------------------------------
Jul-17 11:54:56.602 [main] DEBUG nextflow.Session - Session aborted -- Cause: Is a directory
Jul-17 11:54:56.631 [main] DEBUG nextflow.Session - The following nodes are still active:
  [operator] splitCsv
  [operator] map

Jul-17 11:54:56.648 [main] ERROR nextflow.cli.Launcher - Is a directory
java.io.IOException: Is a directory
        at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at java.base/sun.nio.ch.FileDispatcherImpl.read(FileDispatcherImpl.java:48)
        at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
        at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:245)
        at java.base/sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:223)
        at java.base/sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
        at java.base/sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:107)
        at java.base/sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:101)
        at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.base/java.io.InputStreamReader.read(InputStreamReader.java:181)
        at java.base/java.io.BufferedReader.read1(BufferedReader.java:210)
        at java.base/java.io.BufferedReader.read(BufferedReader.java:287)
        at java.base/java.io.Reader.read(Reader.java:229)
        at org.codehaus.groovy.runtime.IOGroovyMethods.getText(IOGroovyMethods.java:861)
        at org.apache.groovy.nio.extensions.NioExtensions.getText(NioExtensions.java:397)
        at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:200)
        at nextflow.script.IncludeDef.memoizedMethodPriv$loadModule0PathMapSession(IncludeDef.groovy:142)
        at nextflow.script.IncludeDef.access$0(IncludeDef.groovy)
        at nextflow.script.IncludeDef$__clinit__closure1.doCall(IncludeDef.groovy)
        at jdk.internal.reflect.GeneratedMethodAccessor65.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
        at groovy.lang.Closure.call(Closure.java:412)
        at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137)
        at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137)
        at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113)
        at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136)
        at nextflow.script.IncludeDef.loadModule0(IncludeDef.groovy)
        at nextflow.script.IncludeDef.load0(IncludeDef.groovy:114)
        at nextflow.script.IncludeDef$load0$1.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
        at Script_662f11d3.runScript(Script_662f11d3:7)
        at nextflow.script.BaseScript.runDsl2(BaseScript.groovy:170)
        at nextflow.script.BaseScript.run(BaseScript.groovy:217)
        at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:220)
        at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:206)
        at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:200)
        at nextflow.script.IncludeDef.memoizedMethodPriv$loadModule0PathMapSession(IncludeDef.groovy:142)
        at nextflow.script.IncludeDef.access$0(IncludeDef.groovy)
        at nextflow.script.IncludeDef$__clinit__closure1.doCall(IncludeDef.groovy)
        at jdk.internal.reflect.GeneratedMethodAccessor65.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
        at groovy.lang.Closure.call(Closure.java:412)
        at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137)
        at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137)
        at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113)
        at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136)
        at nextflow.script.IncludeDef.loadModule0(IncludeDef.groovy)
        at nextflow.script.IncludeDef.load0(IncludeDef.groovy:114)
        at nextflow.script.IncludeDef$load0$1.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
        at Script_5b6fb2c3.runScript(Script_5b6fb2c3:100)
        at nextflow.script.BaseScript.runDsl2(BaseScript.groovy:170)
        at nextflow.script.BaseScript.run(BaseScript.groovy:217)
        at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:220)
        at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:206)
        at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:200)
        at nextflow.script.IncludeDef.memoizedMethodPriv$loadModule0PathMapSession(IncludeDef.groovy:142)
        at nextflow.script.IncludeDef.access$0(IncludeDef.groovy)
        at nextflow.script.IncludeDef$__clinit__closure1.doCall(IncludeDef.groovy)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
        at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:274)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1035)
        at groovy.lang.Closure.call(Closure.java:412)
        at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.lambda$call$0(Memoize.java:137)
        at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:137)
        at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:113)
        at org.codehaus.groovy.runtime.memoize.Memoize$MemoizeFunction.call(Memoize.java:136)
        at nextflow.script.IncludeDef.loadModule0(IncludeDef.groovy)
        at nextflow.script.IncludeDef.load0(IncludeDef.groovy:114)
        at nextflow.script.IncludeDef$load0$1.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
        at Script_4786426c.runScript(Script_4786426c:29)
        at nextflow.script.BaseScript.runDsl2(BaseScript.groovy:170)
        at nextflow.script.BaseScript.run(BaseScript.groovy:217)
        at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:220)
        at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:212)
        at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:120)
        at nextflow.cli.CmdRun.run(CmdRun.groovy:337)
        at nextflow.cli.Launcher.run(Launcher.groovy:480)
        at nextflow.cli.Launcher.main(Launcher.groovy:639)

~/.nextflow/config

// Forked from https://github.com/nf-core/configs/blob/master/conf/prince.config

def singularityDir = set_singularity_path()

params {
  config_profile_description  = """
                                  WUSTL High Throughput Computing Facility cluster profile provided by nf-core/configs.
                                  Run from your scratch directory, the output files may be large!
                                  Please consider running the pipeline on a compute node the first time, as it will be pulling the docker image, which will be converted into a singularity image, which is heavy on the login node. Subsequent runs can be done on the login node, as the docker image will only be pulled and converted once. By default, the images will be stored in $singularityDir
                                """.stripIndent()
  config_profile_contact      = "Gavin John <gavinjohn@wustl.edu>"
  config_profile_url          = "https://github.com/nf-core/configs/blob/master/docs/wustl-htcf.md"
}

spack {
  enabled = true
}

singularity {
  enabled  = true
  cacheDir = singularityDir
}

process {
  beforeScript = "exec \$( spack load --sh singularity )"
  executor     = "slurm"
}

def set_singularity_path() {
  def lab = System.getenv("LAB")
  if (lab == null){
    System.err.println("")
    System.err.println((String)((char)27) + "[33m" + "WARNING: wustl-htcf.config requires the LAB environment variable to be set. Please add the following line to your .bashrc:" + (String)((char)27) + "[30m")
    System.err.println("""export LAB=\$(id -nG "\$(whoami)" 2> /dev/null | awk 'NF>1{print \$NF}')""")
    System.err.println((String)((char)27) + "[0m")
    return null
  } else {
    return "/ref/$LAB/data/singularity_images_nextflow"
  }
}

System information

gavinjohn@login:/scratch/djslab/gavinjohn $ nextflow -v
nextflow version 22.04.3.5703
gavinjohn@login:/scratch/djslab/gavinjohn $ lscpu 
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         46 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  48
  On-line CPU(s) list:   0-47
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz
    CPU family:          6
    Model:               85
    Thread(s) per core:  2
    Core(s) per socket:  12
    Socket(s):           2
    Stepping:            7
    CPU(s) scaling MHz:  65%
    CPU max MHz:         3200.0000
    CPU min MHz:         1000.0000
    BogoMIPS:            4400.00
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonst
                         op_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefet
                         ch cpuid_fault epb cat_l3 cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f
                          avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke avx512_vnni md_clear flu
                         sh_l1d arch_capabilities
Virtualization features: 
  Virtualization:        VT-x
Caches (sum of all):     
  L1d:                   768 KiB (24 instances)
  L1i:                   768 KiB (24 instances)
  L2:                    24 MiB (24 instances)
  L3:                    33 MiB (2 instances)
NUMA:                    
  NUMA node(s):          2
  NUMA node0 CPU(s):     0-11,24-35
  NUMA node1 CPU(s):     12-23,36-47
Vulnerabilities: [REDACTED]

Hardware: HPC Executor: slurm Container engine: Singularity OS: Unknown linux nf-core/mag version: 2.3.2 or master branch

Pandapip1 commented 1 year ago

Note: I tested this with both the local version (nextflow run /ref/djslab/data/nextflow/nf-core/mag/2_3_2) and the most up-to-date version (nextflow run nf-core/mag).

jfy133 commented 1 year ago

We have fixed this in the upcoming release, you can try running this with the latest version of nextflow (i.e. 23.04) and it should work (as a work around).

jfy133 commented 1 year ago

x-ref https://github.com/nf-core/mag/pull/475

Pandapip1 commented 1 year ago

Might it be worth pinning the x-ref'd issue in the meantime? I searched open issues and didn't find anything (because the issue was closed).

Pandapip1 commented 1 year ago

Actually, it turns out I am running NF 23.04 already, so that fix didn't work.

EDIT: Using the master branch using nextflow run nf-core/mag#master -profile test --outdir test didn't help either.

EDIT 2: I just realized you're using master for stable and dev for development, not master for development and stable for stable. Also, I realized that's not the correct syntax for branch selection.

EDIT 3: dev branch is broken for me too.

gavinjohn@login:/scratch/djslab/gavinjohn $ nextflow run nf-core/mag -r dev -profile test --outdir test
N E X T F L O W  ~  version 22.04.3
Launching `https://github.com/nf-core/mag` [ridiculous_curie] DSL2 - revision: 87761e0b7d [dev]
Cannot find latest version of nf-validation plugin

gavinjohn@login:/scratch/djslab/gavinjohn $ cat .nextflow.log
Jul-18 09:48:50.626 [main] DEBUG nextflow.cli.Launcher - $> nextflow run nf-core/mag -r dev -profile test --outdir test
Jul-18 09:48:50.734 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 22.04.3
Jul-18 09:48:51.905 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/gavinjohn/.nextflow/assets/nf-core/mag/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/mag.git
Jul-18 09:48:51.922 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/gavinjohn/.nextflow/assets/nf-core/mag/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/mag.git
Jul-18 09:48:52.034 [main] WARN  org.eclipse.jgit.util.FS - Exception caught during execution of command '[git, config, --system, --edit]' in '/ref/djslab/software/spack-0.18.1/opt/spack/linux-rocky8-x86_64/gcc-8.5.0/git-2.35.2-j4xfhht2htme6osg7loxtnrcprez4jyg/bin', return code '128', error message 'fatal: Invalid path '/ref/djslab/software/spack-0.18.1/opt/spack/linux-rocky8-x86_64/gcc-8.5.0/git-2.35.2-j4xfhht2htme6osg7loxtnrcprez4jyg/etc': No such file or directory
'
Jul-18 09:48:58.790 [main] DEBUG nextflow.config.ConfigBuilder - Found config home: /home/gavinjohn/.nextflow/config
Jul-18 09:48:58.791 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/gavinjohn/.nextflow/assets/nf-core/mag/nextflow.config
Jul-18 09:48:58.793 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/gavinjohn/.nextflow/config
Jul-18 09:48:58.793 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/gavinjohn/.nextflow/assets/nf-core/mag/nextflow.config
Jul-18 09:48:58.802 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `test`
Jul-18 09:48:58.898 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `test`
Jul-18 09:48:59.359 [main] DEBUG nextflow.plugin.PluginsFacade - Using Default plugins manager
Jul-18 09:48:59.371 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jul-18 09:48:59.372 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jul-18 09:48:59.377 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jul-18 09:48:59.381 [main] DEBUG nextflow.plugin.PluginsFacade - Using Default plugins manager
Jul-18 09:48:59.555 [main] DEBUG nextflow.plugin.PluginsFacade - Using Default plugins manager
Jul-18 09:48:59.556 [main] DEBUG nextflow.plugin.PluginsFacade - Using Default plugins manager
Jul-18 09:49:00.166 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [cfc_dev, ifb_core, denbi_qbic, alice, mjolnir_globe, uppmax, test_binning_entry, incliva, uge, rosalind_uge, test_nothing, lugh, unibe_ibu, vai, czbiohub_aws, jax, ccga_med, scw, tigem, tubingen_apg, google, ipop_up, googlels, eddie, medair, test_host_rm, biowulf, apptainer, bi, bigpurple, sbc_sharc, adcra, cedars, vsc_kul_uhasselt, pawsey_nimbus, ucl_myriad, utd_ganymede, charliecloud, icr_davros, test_ancient_dna, ceres, munin, arm, rosalind, test_binrefinement, hasta, cfc, uzh, test_hybrid_host_rm, test_busco_auto, ebi_codon_slurm, ebc, ku_sund_dangpu, ccga_dx, test_hybrid, crick, marvin, biohpc_gen, shifter, mana, mamba, wehi, awsbatch, imperial, maestro, genotoul, test_no_clipping, abims, janelia, nu_genomics, googlebatch, oist, sahmri, mpcdf, leicester, vsc_ugent, sage, cambridge, podman, ebi_codon, cheaha, xanadu, test, computerome, seg_globe, sanger, dkfz, pasteur, test_full, azurebatch, hki, test_adapterremoval, crukmi, docker, engaging, gis, hypatia, psmn, eva, fgcz, conda, crg, singularity, uw_hyak_pedslabs, prince, utd_sysbio, debug, genouest, cbe, test_bbnorm, phoenix, gitpod, seawulf, fub_curta, uct_hpc, aws_tower, binac]
Jul-18 09:49:00.213 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Jul-18 09:49:00.214 [main] INFO  nextflow.cli.CmdRun - Launching `https://github.com/nf-core/mag` [ridiculous_curie] DSL2 - revision: 87761e0b7d [dev]
Jul-18 09:49:00.215 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/home/gavinjohn/.nextflow/plugins; core-plugins: nf-amazon@1.7.2,nf-azure@0.13.2,nf-console@1.0.3,nf-ga4gh@1.0.3,nf-google@1.1.4,nf-sqldb@0.4.0,nf-tower@1.4.0
Jul-18 09:49:00.217 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins declared=[nf-validation]
Jul-18 09:49:00.224 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jul-18 09:49:00.225 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jul-18 09:49:00.227 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jul-18 09:49:00.235 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Jul-18 09:49:00.236 [main] DEBUG nextflow.plugin.PluginUpdater - Installing plugin nf-validation version: latest
Jul-18 09:49:00.303 [main] ERROR nextflow.cli.Launcher - @unknown
java.lang.IllegalStateException: Cannot find latest version of nf-validation plugin
        at nextflow.plugin.PluginUpdater.load0(PluginUpdater.groovy:241)
        at nextflow.plugin.PluginUpdater.installPlugin(PluginUpdater.groovy:144)
        at nextflow.plugin.PluginUpdater.prepareAndStart(PluginUpdater.groovy:91)
        at nextflow.plugin.PluginsFacade.start(PluginsFacade.groovy:312)
        at nextflow.plugin.PluginsFacade.start(PluginsFacade.groovy:317)
        at nextflow.plugin.PluginsFacade.setup(PluginsFacade.groovy:202)
        at nextflow.plugin.Plugins.setup(Plugins.groovy:39)
        at nextflow.cli.CmdRun.run(CmdRun.groovy:301)
        at nextflow.cli.Launcher.run(Launcher.groovy:480)
        at nextflow.cli.Launcher.main(Launcher.groovy:639)
jfy133 commented 1 year ago

Hmm ok. Let me test and get back to you.

jfy133 commented 1 year ago

Note that you're running 22.04.3 not 23.04 according to your log file.

This works fine on my cluster:

$ > nextflow run nf-core/mag -r dev -profile mpcdf,raven,test --outdir test
N E X T F L O W  ~  version 23.04.2

This fails due to a new plugin in dev that has a new structure not supported by older version of nextflows (the one you are using; the minimum nextflow version in dev will be 23.04 on release)

NXF_VER=22.04.3 nextflow run nf-core/mag -r dev -profile mpcdf,raven,test --outdir test
N E X T F L O W  ~  version 22.04.3
Launching `https://github.com/nf-core/mag` [sharp_wing] DSL2 - revision: 87761e0b7d [dev]
Cannot find latest version of nf-validation plugin

This fails as originally reported (as the older version of Nextflow didn't support modules being loaded simply by directory and has since been fixed in dev regardless)

$ NXF_VER=22.04.3 nextflow run nf-core/mag -r 2.3.2 -profile mpcdf,raven,test --outdir test
Is a directory

 -- Check script '/u/jfellowsy/.nextflow/assets/nf-core/mag/./workflows/../subworkflows/local/gunc_qc.nf' at line: 7 or see '.nextflow.log' file for more details

This is working for me with no issue:

$ nextflow run nf-core/mag -r 2.3.2 -profile mpcdf,raven,test --outdir test
N E X T F L O W  ~  version 23.04.2
Pandapip1 commented 1 year ago

Note that you're running 22.04.3 not 23.04 according to your log file.

I'm blind apparently. Thanks for pointing that out!

jfy133 commented 1 year ago

No worries! Hope it's running now