thaJeztah
commented
3 years ago Thanks for reporting!
First, this is unclear about whether it's only talking about
ENVenvironment variables (as it seems to) or if it also applies toARGbuild arguments.
Yes, this should be better clarified. ENV and ARG both act in the same space, and are both set as environment variables for RUN instructions. The difference is that ENV is persisted in the image, whereas ARG are only available during docker build and their values are discarded afterwards (so they don't persist in the image, unless their value is copied to an ENV).
This seems to directly contradict the absence of
RUNin the "Environment replacement" list. Perhaps it is meant that variable expansion ofENVvariables will be performed by the shell under which theRUNinstruction command executes, rather than by Docker itself. But it is not guaranteed that the shell performs such expansion.
Yes, that is correct; variable expansion in RUN, ENTRYPOINT and CMD instructions is performed by the shell (or other processes that executes the RUN); expansion is performed the moment those instructions run, which, for RUN instructions, is during the docker build, and for ENTRYPOINT and CMD when a container is started from the image. I wrote a reply on https://github.com/moby/moby/issues/42937#issuecomment-945637536, which is also around this issue.
The documentation also gives an example in which an
ARGis used in aRUNinstruction, suggesting that its absence from the "Environment replacement" list above is indeed an omission or something else.
In that example, the shell performs the variable expansion. The RUN in the example;
FROM busybox
ARG SETTINGS
RUN ./run/setup $SETTINGSCan be read as:
SETTINGS=<value of settings ARG> /bin/sh -c './run/setup $SETTINGS'Similarly, the "Environment replacement" list says
FROMsupports expansion of environment variables, but there is a section specifically talking about its behavior with build arguments.
Agreed, that looks incorrect; substitution rules are the same, but it would only expand them if they're set as ARG, and only ARG will be substituted in FROM.
There are some weird edge cases with
ARGandENVlike howENVdeclarations shadow priorARG
If I'm not mistaken, later values should override former values if there's an overlap between ENV and ARG (both setting the same variable name);
FROM busybox
ENV FOO1=env-foo1
ENV FOO2=env-foo2
ENV FOO3=env-foo3
ARG FOO4=arg-foo4-default
ARG FOO5
ENV FOO6=env-foo6
ARG FOO1=arg-foo1-default
ARG FOO2=arg-foo2-default
ARG FOO3
ENV FOO4=env-foo4
ENV FOO5=env-foo5
ARG FOO7=arg-foo7-default
ENV FOO7=copied-from-$FOO7
RUN echo FOO1 is: $FOO1; echo FOO2 is: $FOO2; echo FOO3 is: $FOO3; echo FOO4 is: $FOO4; echo FOO5 is: $FOO5; echo FOO6 is: $FOO6; echo FOO7 is: $FOO7;With BuildKit enabled;
DOCKER_BUILDKIT=1 docker build -t foo1 --build-arg FOO1=cli-foo1 --build-arg FOO4=cli-foo4 --no-cache --progress=plain .
...
#5 0.200 FOO1 is: cli-foo1
#5 0.200 FOO2 is: arg-foo2-default
#5 0.200 FOO3 is: env-foo3
#5 0.200 FOO4 is: env-foo4
#5 0.200 FOO5 is: env-foo5
#5 0.200 FOO6 is: env-foo6
#5 0.200 FOO7 is: copied-from-arg-foo7-defaultHowever, behavior with the classic builder, due to limitations of that builder, is slightly different during the build:
DOCKER_BUILDKIT=0 docker build -t foo2 --build-arg FOO1=cli-foo1 --build-arg FOO4=cli-foo4 --no-cache .
...
FOO1 is: env-foo1
FOO2 is: env-foo2
FOO3 is: env-foo3
FOO4 is: env-foo4
FOO5 is: env-foo5
FOO6 is: env-foo6
FOO7 is: copied-from-arg-foo7-default(Environment variables in the resulting image should be the same in both cases though);
docker image inspect --format='{{json .Config.Env}}' foo2 | jq .
[
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"FOO1=env-foo1",
"FOO2=env-foo2",
"FOO3=env-foo3",
"FOO6=env-foo6",
"FOO4=env-foo4",
"FOO5=env-foo5",
"FOO7=copied-from-arg-foo7-default"
]In general, the recommendation should be to prevent using the same name for ARG and ENV to prevent ambiguity, but agreed that the behavior should be documented more in-depth.
and how
ARGhas to be repeated after aFROM, but before raising these issues the documentation should be much clearer on how to use them in the first place.
As a general thumb of rule, FROM (and COPY --from) can use a ARG that is defined in the global scope (before the first FROM). Instructions within a build stage only have access to ARG defined within that stage (within the lines following the FROM that starts the stage). Some discussion around improving the docs around this can be found in https://github.com/moby/moby/issues/40830#issuecomment-622949605. That discussion does not yet include BuildKit's feature that allows "splitting" a stage, for example, with BuildKit, this is possible;
FROM busybox AS base
ARG FOO=bar
FROM base AS breakpoint1
RUN echo FOO is: $FOO
FROM breakpoint1 AS breakpoint2
RUN echo FOO is: $FOO
Description
The documentation is unclear on a few points about how
ARGbuild arguments andENVenvironment variables are treated differently in terms of variable substitution (interpolation).The Dockerfile reference documentation says:
First, this is unclear about whether it's only talking about
ENVenvironment variables (as it seems to) or if it also applies toARGbuild arguments.Second, it also says:
This seems to directly contradict the absence of
RUNin the "Environment replacement" list. Perhaps it is meant that variable expansion ofENVvariables will be performed by the shell under which theRUNinstruction command executes, rather than by Docker itself. But it is not guaranteed that the shell performs such expansion.The documentation also gives an example in which an
ARGis used in aRUNinstruction, suggesting that its absence from the "Environment replacement" list above is indeed an omission or something else.Similarly, the "Environment replacement" list says
FROMsupports expansion of environment variables, but there is a section specifically talking about its behavior with build arguments.There are some weird edge cases with
ARGandENVlike howENVdeclarations shadow priorARG, and howARGhas to be repeated after aFROM, but before raising these issues the documentation should be much clearer on how to use them in the first place.Output of
docker version:Output of
docker info: