Split this to two issues, since they are definitely separate problems, but this is the follow-up to #28
Our HPC has nextflow so I tried:
nextflow run /groups/adamraikes/git_repos/dmriqc_flow/main.nf --input $PWD/nifti -profile input_qc --output_dir $PWD/test -with-singularity /groups/adamraikes/singularity_images/scilus_1.6.0.sif
N E X T F L O W ~ version 22.10.7
Launching `/groups/adamraikes/git_repos/dmriqc_flow/main.nf` [magical_monod] DSL1 - revision: b23b8d1f03
WARN: Nextflow version 22.10.7 does not match workflow required version: 21.1+ -- Execution will continue, but things may break!
This workflow requires Nextflow version 21.0 or greater -- You are running version 22.10.7
-- Check script '/groups/adamraikes/git_repos/dmriqc_flow/main.nf' at line: 17 or see '.nextflow.log' file for more details
Contrary to its indication, it doesn't keep running:
Jul-18 11:55:13.783 [main] DEBUG nextflow.cli.Launcher - $> nextflow run /groups/adamraikes/git_repos/dmriqc_flow/main.nf --input /xdisk/adamraikes/PD_allo/nifti -profile input_qc --output_dir /xdisk/adamraikes/PD_allo/test -with-singularity /groups/adamraikes/singularity_images/scilus_1.6.0.sif
Jul-18 11:55:13.857 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 22.10.7
Jul-18 11:55:13.873 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/u26/adamraikes/.nextflow/plugins; core-plugins: nf-amazon@1.11.4,nf-azure@0.14.2,nf-codecommit@0.1.2,nf-console@1.0.4,nf-ga4gh@1.0.4,nf-google@1.4.5,nf-tower@1.5.6,nf-wave@0.5.4
Jul-18 11:55:13.885 [main] INFO org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jul-18 11:55:13.886 [main] INFO org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jul-18 11:55:13.889 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jul-18 11:55:13.897 [main] INFO org.pf4j.AbstractPluginManager - No plugins
Jul-18 11:55:13.911 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /groups/adamraikes/git_repos/dmriqc_flow/nextflow.config
Jul-18 11:55:13.913 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /groups/adamraikes/git_repos/dmriqc_flow/nextflow.config
Jul-18 11:55:13.953 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `input_qc`
Jul-18 11:55:14.621 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [tractoflow_qc_light, cbrain, input_qc, disconets_qc, tractoflow_qc_all, rbx_qc]
Jul-18 11:55:14.646 [main] DEBUG nextflow.config.ConfigBuilder - Enabling execution in Singularity container as requested by command-line option `-with-singularity null`
Jul-18 11:55:14.660 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=1 by probing script field
Jul-18 11:55:14.680 [main] INFO nextflow.cli.CmdRun - Launching `/groups/adamraikes/git_repos/dmriqc_flow/main.nf` [magical_monod] DSL1 - revision: b23b8d1f03
Jul-18 11:55:14.680 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Jul-18 11:55:14.680 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Jul-18 11:55:14.688 [main] DEBUG nextflow.secret.LocalSecretsProvider - Secrets store: /home/u26/adamraikes/.nextflow/secrets/store.json
Jul-18 11:55:14.691 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@574b7f4a] - activable => nextflow.secret.LocalSecretsProvider@574b7f4a
Jul-18 11:55:14.732 [main] DEBUG nextflow.Session - Session UUID: ab0c4fc0-1004-4141-a61d-f109ea1d88b3
Jul-18 11:55:14.733 [main] DEBUG nextflow.Session - Run name: magical_monod
Jul-18 11:55:14.733 [main] DEBUG nextflow.Session - Executor pool size: 48
Jul-18 11:55:14.742 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=144; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jul-18 11:55:14.767 [main] DEBUG nextflow.cli.CmdRun -
Version: 22.10.7 build 5853
Created: 18-02-2023 20:32 UTC (13:32 MDT)
System: Linux 3.10.0-1160.114.2.el7.x86_64
Runtime: Groovy 3.0.13 on OpenJDK 64-Bit Server VM 18.0.2.1+1-1
Encoding: UTF-8 (ANSI_X3.4-1968)
Process: 56395@r1u10n1.puma.hpc.arizona.edu [10.141.16.29]
CPUs: 48 - Mem: 240 GB (236.9 GB) - Swap: 0 (0)
Jul-18 11:55:14.800 [main] DEBUG nextflow.Session - Work-dir: /xdisk/adamraikes/PD_allo/work [nfs]
Jul-18 11:55:14.800 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /groups/adamraikes/git_repos/dmriqc_flow/bin
Jul-18 11:55:14.809 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Jul-18 11:55:14.817 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Jul-18 11:55:14.880 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Jul-18 11:55:14.888 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 49; maxThreads: 1000
Jul-18 11:55:14.974 [main] DEBUG nextflow.Session - Session start
Jul-18 11:55:15.655 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Jul-18 11:55:15.661 [main] WARN nextflow.Session - Nextflow version 22.10.7 does not match workflow required version: 21.1+ -- Execution will continue, but things may break!
Jul-18 11:55:15.669 [main] DEBUG nextflow.Session - Session aborted -- Cause: This workflow requires Nextflow version 21.0 or greater -- You are running version 22.10.7
Jul-18 11:55:15.677 [main] ERROR nextflow.cli.Launcher - @unknown
nextflow.exception.ProcessUnrecoverableException: This workflow requires Nextflow version 21.0 or greater -- You are running version 22.10.7
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483)
at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:72)
at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:59)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:263)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:277)
at nextflow.Nextflow.error(Nextflow.groovy:262)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:577)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite$StaticMetaMethodSiteNoUnwrap.invoke(StaticMetaMethodSite.java:131)
at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:100)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:55)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:217)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:231)
at Script_7d57c9e2.runScript(Script_7d57c9e2:17)
at nextflow.script.BaseScript.runDsl1(BaseScript.groovy:164)
at nextflow.script.BaseScript.run(BaseScript.groovy:217)
at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:230)
at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:225)
at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:131)
at nextflow.cli.CmdRun.run(CmdRun.groovy:354)
at nextflow.cli.Launcher.run(Launcher.groovy:487)
at nextflow.cli.Launcher.main(Launcher.groovy:646)
Split this to two issues, since they are definitely separate problems, but this is the follow-up to #28
Our HPC has
nextflow
so I tried:Contrary to its indication, it doesn't keep running: