aws / amazon-genomics-cli

https://aws.github.io/amazon-genomics-cli/
Apache License 2.0
147 stars 83 forks source link

PATH overwritten on debian-based images using miniwdl #598

Open hkeward opened 1 year ago

hkeward commented 1 year ago

Describe the Bug The command that miniwdl executes in the batch job:

["/bin/bash","-ec","cd /mnt/efs/<wf_id>/1/call-say_hello/work\nexit_code=0\nbash -l ../command >> ../stdout.txt 2> >(tee -a ../stderr.txt >&2) || exit_code=$?\nexit $exit_code"]

More readably:

/bin/bash -ec "cd /mnt/efs/<wf_id>/1/call-say_hello/work
exit_code=0
bash -l ../command >> ../stdout.txt 2> >(tee -a ../stderr.txt >&2) || exit_code=$?
exit $exit_code"

The comand is executing as a login shell (bash -l ../command); this means that /etc/profile is first sourced, which overwrites the PATH to /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin. If I've extended or altered the PATH in the docker image running the task, this change will be overwritten by the PATH in /etc/profile, meaning that the task is potentially unable to find executables.

This has been fixed in release 0.9.0 of miniwdl-aws.

Steps to Reproduce

Dockerfile (available as hkeward/path:1); any Docker image based on debian should work.


FROM python:3.9-buster

ENV PATH "${PATH}:/opt"


> test.wdl
```wdl
version 1.0

workflow test_path {
  call echo_path

  output {
    String path = echo_path.path
  }
}

task echo_path {
  command <<<
    echo "$PATH"
  >>>

  output {
    String path = read_string(stdout())
  }

  runtime {
    docker: "hkeward/path:1"
    cpu: 1
    memory: "1 GB"
  }
}

Run this using a miniwdl engine via agc.

Expected Behavior

The output path should be /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt.

Actual Behavior

The actual path output is /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin, which is the path defined in /etc/profile in debian-based images.

Additional Context

Operating System: Debian GNU/Linux 11 (bullseye) AGC Version: 1.6.0 Was AGC setup with a custom bucket: Yes Was AGC setup with a custom VPC: Yes