Nextflow does not use the specified containers

Bug report

Expected behavior and actual behavior

Whenever I try to run any pipeline (e.g. nf-core ones) with Docker or Singularity it does not use the container to run the pipeline. Hence, all commands of the processes are not found and the pipeline just fails. I can run the containers outside of any Nextflow pipelines without any problems and the commands are found (as expected). Example:

⋊> /tmp nextflow run nf-core/rnaseq -profile test,singularity                                                                                                               (spapros) 14:41:57
N E X T F L O W  ~  version 21.04.1
Launching `nf-core/rnaseq` [evil_hawking] - revision: b3ff92bc54 [master]

------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/rnaseq v3.2
------------------------------------------------------

...

Cannot run program "hostname": error=2, No such file or directory

 -- Check script '/home/zeth/.nextflow/assets/nf-core/rnaseq/main.nf' at line: 35 or see '.nextflow.log' file for more details

Steps to reproduce the problem

nextflow run nf-core/rnaseq -profile test,singularity

Program output

Jul-05 14:45:02.520 [main] DEBUG nextflow.cli.Launcher - $> nextflow run nf-core/rnaseq -profile test,singularity
Jul-05 14:45:02.637 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 21.04.1
Jul-05 14:45:03.854 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/zeth/.nextflow/assets/nf-core/rnaseq/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/rnaseq.git
Jul-05 14:45:03.865 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/zeth/.nextflow/assets/nf-core/rnaseq/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/rnaseq.git
Jul-05 14:45:03.964 [main] INFO  nextflow.cli.CmdRun - Launching `nf-core/rnaseq` [extravagant_laplace] - revision: b3ff92bc54 [master]
Jul-05 14:45:04.639 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/zeth/.nextflow/assets/nf-core/rnaseq/nextflow.config
Jul-05 14:45:04.640 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/zeth/.nextflow/assets/nf-core/rnaseq/nextflow.config
Jul-05 14:45:04.647 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `test,singularity`
Jul-05 14:45:05.150 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jul-05 14:45:05.151 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jul-05 14:45:05.154 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jul-05 14:45:05.888 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [cfc_dev, ifb_core, denbi_qbic, genotoul, uppmax, abims, imperial_mb, oist, mpcdf, podman, czbiohub_aws, jax, ccga_med, test, google, kraken, seg_globe, shh, sanger, pasteur, test_full, eddie, bi, bigpurple, docker, gis, eva, utd_ganymede, charliecloud, conda, singularity, icr_davros, munin, prince, hebbe, cfc, utd_sysbio, uzh, debug, genouest, cbe, ebc, ccga_dx, crick, phoenix, biohpc_gen, shifter, awsbatch, uct_hpc, imperial, aws_tower, binac]
Jul-05 14:45:05.900 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/home/zeth/.nextflow/plugins
Jul-05 14:45:05.902 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Jul-05 14:45:05.902 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins local root: .nextflow/plr/empty
Jul-05 14:45:05.905 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jul-05 14:45:05.906 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jul-05 14:45:05.907 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Jul-05 14:45:05.915 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Jul-05 14:45:05.961 [main] DEBUG nextflow.Session - Session uuid: 3fd8611a-7f82-4afa-9572-d9ab6659873c
Jul-05 14:45:05.961 [main] DEBUG nextflow.Session - Run name: extravagant_laplace
Jul-05 14:45:05.961 [main] DEBUG nextflow.Session - Executor pool size: 8
Jul-05 14:45:05.971 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 21.04.1 build 5556
  Created: 14-05-2021 15:20 UTC (17:20 CEST)
  System: Linux 5.12.14-arch1-1
  Runtime: Groovy 3.0.7 on OpenJDK 64-Bit Server VM 1.8.0_152-release-1056-b12
  Encoding: UTF-8 (ANSI_X3.4-1968)
  Process: 482309@master [192.168.0.49]
  CPUs: 8 - Mem: 15.3 GB (256.3 MB) - Swap: 12 GB (12 GB)
Jul-05 14:45:05.984 [main] DEBUG nextflow.Session - Work-dir: /tmp/work [tmpfs]
Jul-05 14:45:05.998 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Jul-05 14:45:06.010 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Jul-05 14:45:06.412 [main] DEBUG nextflow.Session - Session start invoked
Jul-05 14:45:06.415 [main] DEBUG nextflow.trace.TraceFileObserver - Flow starting -- trace file: /tmp/results/pipeline_info/execution_trace_2021-07-05_14-45-05.txt
Jul-05 14:45:06.423 [main] DEBUG nextflow.Session - Using default localLib path: /home/zeth/.nextflow/assets/nf-core/rnaseq/lib
Jul-05 14:45:06.427 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/zeth/.nextflow/assets/nf-core/rnaseq/lib
Jul-05 14:45:06.427 [main] DEBUG nextflow.Session - Adding to the classpath library: /home/zeth/.nextflow/assets/nf-core/rnaseq/lib/nfcore_external_java_deps.jar
Jul-05 14:45:07.394 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Jul-05 14:45:07.658 [main] INFO  nextflow.Nextflow - 

-[2m----------------------------------------------------[0m-
                                        [0;32m,--.[0;30m/[0;32m,-.[0m
[0;34m        ___     __   __   __   ___     [0;32m/,-._.--~'[0m
[0;34m  |\ | |__  __ /  ` /  \ |__) |__         [0;33m}  {[0m
[0;34m  | \| |       \__, \__/ |  \ |___     [0;32m\`-._,-`-,[0m
                                        [0;32m`._,._,'[0m
[0;35m  nf-core/rnaseq v3.2[0m
-[2m----------------------------------------------------[0m-
[1mCore Nextflow options[0m
  [0;34mrevision                  : [0;32mmaster[0m
  [0;34mrunName                   : [0;32mextravagant_laplace[0m
  [0;34mcontainerEngine           : [0;32msingularity[0m
  [0;34mlaunchDir                 : [0;32m/tmp[0m
  [0;34mworkDir                   : [0;32m/tmp/work[0m
  [0;34mprojectDir                : [0;32m/home/zeth/.nextflow/assets/nf-core/rnaseq[0m
  [0;34muserName                  : [0;32mzeth[0m
  [0;34mprofile                   : [0;32mtest,singularity[0m
  [0;34mconfigFiles               : [0;32m/home/zeth/.nextflow/assets/nf-core/rnaseq/nextflow.config[0m

[1mInput/output options[0m
  [0;34minput                     : [0;32mhttps://raw.githubusercontent.com/nf-core/test-datasets/rnaseq/samplesheet/v3.1/samplesheet_test.csv[0m

[1mUMI options[0m
  [0;34mumitools_bc_pattern       : [0;32mNNNN[0m

[1mReference genome options[0m
  [0;34mfasta                     : [0;32mhttps://github.com/nf-core/test-datasets/raw/rnaseq/reference/genome.fa[0m
  [0;34mgtf                       : [0;32mhttps://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gtf.gz[0m
  [0;34mgff                       : [0;32mhttps://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gff.gz[0m
  [0;34mtranscript_fasta          : [0;32mhttps://github.com/nf-core/test-datasets/raw/rnaseq/reference/transcriptome.fasta[0m
  [0;34madditional_fasta          : [0;32mhttps://github.com/nf-core/test-datasets/raw/rnaseq/reference/gfp.fa.gz[0m
  [0;34mstar_index                : [0;32mhttps://github.com/nf-core/test-datasets/raw/rnaseq/reference/star.tar.gz[0m
  [0;34mhisat2_index              : [0;32mhttps://github.com/nf-core/test-datasets/raw/rnaseq/reference/hisat2.tar.gz[0m
  [0;34mrsem_index                : [0;32mhttps://github.com/nf-core/test-datasets/raw/rnaseq/reference/rsem.tar.gz[0m
  [0;34msalmon_index              : [0;32mhttps://github.com/nf-core/test-datasets/raw/rnaseq/reference/salmon.tar.gz[0m

[1mAlignment options[0m
  [0;34mpseudo_aligner            : [0;32msalmon[0m

[1mInstitutional config options[0m
  [0;34mconfig_profile_name       : [0;32mTest profile[0m
  [0;34mconfig_profile_description: [0;32mMinimal test dataset to check pipeline function[0m

[1mMax job request options[0m
  [0;34mmax_cpus                  : [0;32m2[0m
  [0;34mmax_memory                : [0;32m6 GB[0m
  [0;34mmax_time                  : [0;32m6h[0m

!! Only displaying parameters that differ from the pipeline defaults !!
-[2m----------------------------------------------------[0m-
If you use nf-core/rnaseq for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.1400710

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/rnaseq/blob/master/CITATIONS.md
-[2m----------------------------------------------------[0m-
Jul-05 14:45:07.662 [main] DEBUG nextflow.Session - Session aborted -- Cause: Cannot run program "hostname": error=2, No such file or directory
Jul-05 14:45:07.672 [main] ERROR nextflow.cli.Launcher - Cannot run program "hostname": error=2, No such file or directory
java.io.IOException: Cannot run program "hostname": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
    at java.lang.Runtime.exec(Runtime.java:620)
    at java.lang.Runtime.exec(Runtime.java:450)
    at java.lang.Runtime.exec(Runtime.java:347)
    at org.codehaus.groovy.runtime.ProcessGroovyMethods.execute(ProcessGroovyMethods.java:538)
    at org.codehaus.groovy.runtime.dgm$966.invoke(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:247)
    at org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:130)
    at NfcoreTemplate.hostName(NfcoreTemplate.groovy:27)
    at NfcoreTemplate$hostName$3.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:157)
    at WorkflowMain.initialise(WorkflowMain.groovy:72)
    at WorkflowMain$initialise$0.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:157)
    at Script_4c643811.runScript(Script_4c643811:35)
    at nextflow.script.BaseScript.runDsl2(BaseScript.groovy:170)
    at nextflow.script.BaseScript.run(BaseScript.groovy:200)
    at nextflow.script.ScriptParser.runScript(ScriptParser.groovy:221)
    at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:212)
    at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:120)
    at nextflow.cli.CmdRun.run(CmdRun.groovy:302)
    at nextflow.cli.Launcher.run(Launcher.groovy:475)
    at nextflow.cli.Launcher.main(Launcher.groovy:657)
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    ... 29 common frames omitted

Environment

Nextflow version: [?] 21.04.1
Java version: [?] 8
Operating system: [macOS, Linux, etc]: Arch Linux
Bash version: (use the command $SHELL --version) Fish 3.2.2

Additional context

Running pipelines with Conda works, but containers (Docker, Singularity) do not. Again, my container setup looks fine to me because I can run the containers interactively without any issues.

nextflow-io / nextflow