docker / cli

The Docker CLI
Apache License 2.0
4.75k stars 1.88k forks source link

--build-args/ARG does not overwrite ENV variables of the same name from a base image #3344

Open theelderbeever opened 2 years ago

theelderbeever commented 2 years ago

Description

ARG variables of the same name as an environment variable set in a base image do not overwrite the value of the variable during build time. Is it intentional for the primary environment to take precedence over the temporary environment? This is especially irritating when a base image sets a proxy value to empty and the child image needs to temporarily set a build environment that uses a different proxy value than the parent image.

Steps to reproduce the issue:

  1. Build a parent image and set a variable with ENV
  2. Base a child image off the built parent and apply an ARG of the same name as the parent ENV var.
  3. Echo the ARG in a dockerfile RUN directive in the child
  4. Build the child. Note that the ENV variable of the parent is echo'd not the temporary ARG value

Describe the results you received:

Create parent dockerfile and set an ENV variable.

# parent.dockerfile

FROM alpine:latest

ENV HELLO="World"

Build the parent image

❯ docker build -t parent:latest -f parent.dockerfile .

Create a child dockerfile and set a build-arg with the same name as the ENV var from parent.

# child.dockerfile

FROM parent:latest

ARG HELLO

RUN echo "HELLO=${HELLO}"

Build the child image.

❯ docker build -t child:latest --build-arg HELLO=Earth -f child.dockerfile --no-cache .
Sending build context to Docker daemon  3.072kB
Step 1/4 : FROM parent:latest
 ---> b319a2900116
Step 2/4 : ARG HELLO
 ---> Running in df26b87ef593
Removing intermediate container df26b87ef593
 ---> 037665231ae3
Step 3/4 : RUN echo "HELLO=${HELLO}"
 ---> Running in b79b3944af01
HELLO=World
Removing intermediate container b79b3944af01
 ---> 6772fb18124d
Step 4/4 : CMD [ "echo", "HELLO=${HELLO}" ]
 ---> Running in 8f7fc5bc845b
Removing intermediate container 8f7fc5bc845b
 ---> fdaed33c1026
Successfully built fdaed33c1026
Successfully tagged child:latest

Result: In Step 3/4 the value echo'd is that of the original ENV var in the parent.

Describe the results you expected: I would expect that the temporary environment would take precedence over the core environment during build and echo would produce HELLO=Earth in the example above.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:52:10 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
  compose: Docker Compose (Docker Inc., v2.0.0-rc.3)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 6
  Running: 3
  Paused: 0
  Stopped: 3
 Images: 67
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.774GiB
 Name: docker-desktop
 ID: BSL6:V5BU:UHMQ:5ZQ7:QELY:VD6D:A7U5:434O:7WSO:FHHJ:SAMI:UONQ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http://proxy-lmi.global.lmco.com:80
 HTTPS Proxy: http://proxy-lmi.global.lmco.com:80
 No Proxy: 127.0.0.10/8,localhost,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.lmco.com
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

thaJeztah commented 2 years ago

This is a known limitation of the classic (non-buildkit) builder; with work in progress to make BuildKit the default builder (for Docker Desktop, this is already the case, but not yet for Linux installs), I'm not sure if this will be addressed.

Here's the example that's provided above, but with BuildKit used;

DOCKER_BUILDKIT=1 docker build -t parent:latest -f parent.dockerfile .
[+] Building 0.1s (5/5) FINISHED
 => [internal] load build definition from parent.dockerfile                                   0.0s
 => => transferring dockerfile: 83B                                                           0.0s
 => [internal] load .dockerignore                                                             0.0s
 => => transferring context: 2B                                                               0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                              0.0s
 => CACHED [1/1] FROM docker.io/library/alpine:latest                                         0.0s
 => exporting to image                                                                        0.0s
 => => exporting layers                                                                       0.0s
 => => writing image sha256:04ee7e261e91f63c023b9d72dc33890c2ec33845ef07e3d8ba9b5bb8484ec256  0.0s
 => => naming to docker.io/library/parent:latest                                              0.0s

Then, building the "child" image (I'm using --progress=plain to show the output of each step;

DOCKER_BUILDKIT=1 docker build -t child:latest --progress=plain  --build-arg HELLO=Earth -f child.dockerfile --no-cache .
#1 [internal] load build definition from child.dockerfile
#1 transferring dockerfile:
#1 transferring dockerfile: 101B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/parent:latest
#3 DONE 0.0s

#4 [1/2] FROM docker.io/library/parent:latest
#4 CACHED

#5 [2/2] RUN echo "HELLO=Earth"
#5 0.304 HELLO=Earth
#5 DONE 0.3s

#6 exporting to image
#6 exporting layers 0.1s done
#6 writing image sha256:6d2aa4b9d3e6a05b1341f66a5af77631e7b21bc43aad18c50491d565d4160074
#6 writing image sha256:6d2aa4b9d3e6a05b1341f66a5af77631e7b21bc43aad18c50491d565d4160074 done
#6 naming to docker.io/library/child:latest done
#6 DONE 0.1s

If possible, I'd recommend using BuildKit for your builds (easiest way to enable that is to set DOCKER_BUILDKIT=1 in your shell, or add it to your shell's profile)

theelderbeever commented 2 years ago

Ah understood and thank you. Unfortunately, due to the environment we use specifically we can't use build kit because of the x503 self signed certificate issue with build kit and private registries. Good to know this particular issue is solved though going forward.

thaJeztah commented 2 years ago

Unfortunately, due to the environment we use specifically we can't use build kit because of the x503 self signed certificate issue with build kit and private registries.

Hmm.. good one; I recall this being more of an issue when using the docker-container driver, but perhaps I'm mistaken; @crazy-max can you fill me in on that one? (are there still issues to fix for that?)