environment variables with name that is a substring of another variable expanded incorrectly

Version Info:

Charliecloud: 0.34 (loaded using Lmod)
OS: Red Hat Enterprise Linux

While working on a minimal project I'm using to debug some MPI-in-container issues, I noticed some oddities in Charliecloud's behavior which occur during the build process when expanding variables in a Dockerfile.

Located at the bottom of this issue is a Dockerfile definition representing a minimal build for debugging components in a containerized MPI installation. Inside, a number of environment variables are first defined and later referenced in the build steps. Note that some like ENV PATH="<path-to-something>:$PATH" are defined merely to simplify the use of binaries and libraries later on within the container.

Attached are two different build logs. The first is from building the container locally with Docker (version 24.0.5 on Darwin/MacOS Big sur) and the second results from taking the same action with Charliecloud in a HPC cluster environment.

In the Charliecloud build, the variables either do not expand or in some cases the expansion is performed prematurely, causing the substitution of incorrect values. For an example, in this Dockerfile when Charliecloud attempts to expand ${PATH_INSTALL_BASE}, it instead expands $PATH and appends_INSTALL_BASE} to the end of the expansion.

In addition to the early expansion, I noticed that all of environment variables defined in the build file were not present in the resulting environment built with Charliecloud (for a full Charliecloud build, remove all references to the variables in commands below their definitions and replace with the appropriate static string content).

Dockerfile DEF:


  1 FROM ubuntu:20.04
  2 ENV CMPI_VERSION=openmpi-4.1.6
  3 ENV PATH_INSTALL_BASE=/custom
  4 ENV PATH_INSTALL_MPI=${PATH_INSTALL_BASE}/mpi-installation
  5 ENV PATH_CMPI_BIN=${PATH_INSTALL_MPI}/bin
  6 ENV CMPI_VERSION=openmpi-4.1.6
  7
  8 RUN mkdir $PATH_INSTALL_BASE
  9 WORKDIR $PATH_INSTALL_BASE
 10 RUN apt-get update
 11 RUN apt-get install -y make git wget tar
 12 RUN apt-get install -y gcc g++ gfortran
 13 RUN apt-get install -y libmunge-dev

 14 RUN apt-get install -y m4=1.4.* autoconf=2.69* automake=1:1.16*
 15 RUN apt-get install -y libtool=2.4.6* flex=2.6.4*
 16
 17 RUN wget https://download.open-mpi.org/release/open-mpi/v4.1/${CMPI_VERSION}.tar.gz
 18 RUN tar -xzvf ${CMPI_VERSION}.tar.gz
 19 WORKDIR ${PATH_INSTALL_BASE}/${CMPI_VERSION}
 20 RUN ./configure --prefix=${PATH_INSTALL_MPI}
 21 RUN make all
 22 RUN make install
 23 ENV PATH="${PATH_CMPI_BIN}:${PATH}"
 24 RUN mpirun --version

build.log.docker.md build.log.charliecloud.md

hpc / charliecloud

environment variables with name that is a substring of another variable expanded incorrectly #1751