env.PATH not working when executor is AWS Batch

[sralchemab]

Bug report

Expected behavior and actual behavior

When I set then env config scope for the PATH variable, its value is passed properly when I use "local" executor (with and without docker). However, when I use AWS Batch as executor, the PATH variable never gets the provided value while other variables are properly passed.

Steps to reproduce the problem

It can be tested with a minimal task on a main.nf file and the following values on the nextflow.config file:

process.executor = 'awsbatch'
aws.batch.cliPath = '/home/ec2-user/miniconda/bin/aws'
env {
    MY_TEST_VAR='hello world'
    ANOTHER_TEST="goodbye, test"
    PATH = "/home/ec2-user/miniconda/bin:$PATH"
    AFTER_PATH="${aws.batch.cliPath}"
}

Program output

Here's the command.run file (I had to rename it to txt to attach it here). You will notice on the nxf_main function the following values:

    export PATH=$PWD/nextflow-bin:$PATH
    export MY_TEST_VAR="hello world"
    export ANOTHER_TEST="goodbye, test"
    export AFTER_PATH="/home/ec2-user/miniconda/bin/aws"

And when I run the env command on the task itself, I get the following values (some vars were removed):

NXF_TASK_WORKDIR=/tmp/nxf.wmcLzuFCXL
PWD=/tmp/nxf.wmcLzuFCXL
_=/usr/bin/env
HOME=/root
MY_TEST_VAR=hello world
ANOTHER_TEST=goodbye, test
PATH=//nextflow-bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
AFTER_PATH=/home/ec2-user/miniconda/bin/aws
OLDPWD=/

Environment

• Nextflow version: 23.10.0
• Operating system: Amazon Linux 2023

[bentsherman]

I think the problem is that "/home/ec2-user/miniconda/bin:$PATH" will resolve $PATH immediately rather than in the task environment. Instead you should do "/home/ec2-user/miniconda/bin:\$PATH"

[sralchemab]

Tried that as well. Getting same result.

[bentsherman]

How about single quotes: '/home/ec2-user/miniconda/bin:$PATH'

[sralchemab]

Same thing.