`NXF_FILE_ROOT` not properly resolved for relative paths

[artur-baranowski-aa]

Bug report

Unsure if I'm using this wrong, any help would be appreciated.

Expected behavior and actual behavior

According to the documentation and https://github.com/nextflow-io/nextflow/pull/3942, setting the NXF_FILE_ROOT variable should resolve some relative path relative_path as ${NXF_FILE_ROOT}/relative_path.

When running the workflow below, the files are generated in the specified directory under ${NXF_FILE_ROOT}, but nextflow doesn't recognize them.

Steps to reproduce the problem

Set some path prefix in the environment variable NXF_FILE_ROOT in the launching environment of Nextflow. Execute this minimal workflow:

#!/usr/bin/env nextflow

nextflow.enable.dsl=2

params.n = 10

Channel.from(1..params.n)
    .set { numbers }

process generateFiles {
    input:
    val number

    output:
    file("file_${number}.txt")

    script:
    """
    touch ${NXF_FILE_ROOT}/file_${number}.txt
    """
}

workflow {
    Channel.from(1..params.n) | generateFiles
}

Program output

Files are present under NXF_FILE_ROOT, however nextflow does not recognize them:

 N E X T F L O W   ~  version 24.04.4

Launching `workflows/example/main.nf` [dreamy_laplace] DSL2 - revision: b4399171c7

executor >  local (10)
[3b/c6248c] process > generateFiles (3) [  0%] 0 of 10
executor >  local (10)
[3b/c6248c] process > generateFiles (3) [100%] 10 of 10, failed: 10 ✘
ERROR ~ Error executing process > 'generateFiles (6)'

Caused by:
  Missing output file(s) `file_6.txt` expected by process `generateFiles (6)`

Command executed:

  touch /home/data/file_6.txt

Command exit status:
  0

Command output:
  (empty)

Work dir:
  /home/nextflow/work/9b/d7847fc9ae6c91d490a121168a4da6

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details

Environment

• Nextflow version: 24.04.4
• Java version: openjdk 21-internal 2023-09-19
• Operating system: Ubuntu 24.04 LTS
• Bash version: 5.2.21

[artur-baranowski-aa]

Is there a workaround for this issue? I need to recognize files that have been stored on a shared network drive.

[pditommaso]

You need to specify it as a regular input file

[pditommaso]

For completeness, the NXF_FILE_ROOT is a variable used by the nextflow runtime to resolve relative paths when using file of channel.fromPath. It's not meant to be used at task level.

Even more, a core nextflow pattern is to NOT used absolute path in the task script.

[pditommaso]

Closing this. Feel free to comment below if needed