search

docker / compose

Define and run multi-container applications with Docker
https://docs.docker.com/compose/
Apache License 2.0
33.8k stars 5.2k forks source link

[BUG] Proxy not applied when building image (First Launch / No Cache) #11218

Open LoSunny opened 10 months ago

LoSunny commented 10 months ago

Description

The proxy will not be applied when pulled an image (new / old, as long as it's metadata isn't in cache)

Steps To Reproduce

Error reproduce

  1. Create a docker-compose.yml
  2. Create a service which link to a custom Dockerfile
  3. Build a image using docker compose build

Temporary workaround to solve the issue

Aim: Load the metadata in cache Here is my file structure

[admin@wazuh-01 tmp]$ ls
docker-compose.yml  Dockerfile
[admin@wazuh-01 tmp]$ cat docker-compose.yml
services:
  test:
    build: .
[admin@wazuh-01 tmp]$ cat Dockerfile
FROM python:3

RUN echo "HTTP_PROXY=$HTTP_PROXY"
  1. As docker pull will use the proxy, we can force it to pull the image, i.e. docker pull <your image>
  2. Run docker compose build and it will return success Full sample log 
    
[admin@wazuh-01 tmp]$ sudo docker compose build --progress=plain --no-cache
--progress is a global compose flag, better use `docker compose --progress xx build ...#0 building with "default" instance using docker driver

1 [test internal] load .dockerignore

1 transferring context: 2B done

1 DONE 0.1s

2 [test internal] load build definition from Dockerfile

2 transferring dockerfile: 144B done

2 DONE 0.1s

3 [test internal] load metadata for docker.io/library/python:3

3 ERROR: failed to authorize: DeadlineExceeded: failed to fetch anonymous token: Get "https://auth.docker.io/token?scope=repository%3Alibrary%2Fpython%3Apull&service=registry.docker.io": dial tcp 3.219.239.5:443: i/o timeout

[test internal] load metadata for docker.io/library/python:3:

failed to solve: DeadlineExceeded: DeadlineExceeded: DeadlineExceeded: python:3: failed to authorize: DeadlineExceeded: failed to fetch anonymous token: Get "https://auth.docker.io/token?scope=repository%3Alibrary%2Fpython%3Apull&service=registry.docker.io": dial tcp 3.219.239.5:443: i/o timeout

[admin@wazuh-01 tmp]$ sudo docker pull python:3 3: Pulling from library/python Digest: sha256:31ceea009f42df76371a8fb94fa191f988a25847a228dbeac35b6f8d2518a6ef Status: Image is up to date for python:3 docker.io/library/python:3

[admin@wazuh-01 tmp]$ sudo docker compose build --progress=plain --no-cache --progress is a global compose flag, better use `docker compose --progress xx build ...#0 building with "default" instance using docker driver

1 [test internal] load build definition from Dockerfile

1 transferring dockerfile: 87B 0.0s

1 transferring dockerfile: 144B 0.0s done

1 DONE 0.1s

2 [test internal] load .dockerignore

2 transferring context: 2B done

2 DONE 0.1s

3 [test internal] load metadata for docker.io/library/python:3

3 DONE 0.0s

4 [test 1/2] FROM docker.io/library/python:3

4 DONE 0.4s

5 [test 2/2] RUN echo "HTTP_PROXY=$HTTP_PROXY"

5 0.410 HTTP_PROXY=http://192.168.140.92:8118

5 DONE 0.5s

6 [test] exporting to image

6 exporting layers 0.0s done

6 writing image sha256:359694de5b1dd714c235746eccbeab512695cd486ff80fe10deff3a5274c32e1 done

6 naming to docker.io/library/tmp-test 0.0s done

6 DONE 0.1s


### Compose Version

I have tried two docker compose version, one by the system `dnf`, one installed from GitHub
```Text
[root@wazuh-01 docker]# docker compose version
Docker Compose version v2.21.0
[root@wazuh-01 docker]# docker-compose version
-bash: docker-compose: command not found
[root@wazuh-01 docker]# /root/.docker/cli-plugins/docker-compose-linux-x86_64 version
Docker Compose version v2.23.2

For simplicity, all commands will be referenced by docker compose ... instead of /root/.docker/cli-plugins/docker-compose-linux-x86_64 ...

Docker Environment

[root@wazuh-01 docker]# docker info
Client: Docker Engine - Community
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
WARNING: Plugin "/root/.docker/cli-plugins/docker-compose-linux-x86_64" is not valid: plugin candidate "compose-linux-x86_64" did not match "^[a-z][a-z0-9]*$"

Server:
 Containers: 8
  Running: 6
  Paused: 0
  Stopped: 2
 Images: 21
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc version: v1.1.9-0-gccaecfc
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.14.0-284.30.1.el9_2.x86_64
 Operating System: Rocky Linux 9.2 (Blue Onyx)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.36GiB
 Name: wazuh-01
 ID: 7772d745-3a18-450a-b6db-08c2b740b121
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http://192.168.140.92:8118
 HTTPS Proxy: http://192.168.140.92:8118
 No Proxy: localhost,127.0.0.1
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

glours commented 10 months ago

Hello @LoSunny Do you have the same issue when using docker build? Did you check your proxy is well configured to work with Docker? You can check the documentation here. A workaround could be passing the HTTP_PROXY as a build argument in your command like this docker compose build --build-arg HTTP_PROXY="http://proxy.example.com:3128" .

LoSunny commented 10 months ago

@glours (UPDATED) No, I don't have the same issue when using docker build. If you look at the temporary workaround section above, in the successful attempt of docker compose build ..., step no. 5, it will echo the HTTP_PROXY variable, which states that I have successfully pass the variable through the ~/.docker/config.json file already. Even if I override it with --build-args, it produced the same result

ndeloof commented 10 months ago

If the same issue applies to a plain docker build then this is not a compose bug. Please report on https://github.com/docker/buildx

LoSunny commented 10 months ago

@ndeloof This issue does NOT apply to plain docker build, docker build will be able to download the metadata, but docker compose doesn't. In the temporary workaround section, I indeed use docker build to pull the metadata to cache for docker compose to resolve it

ndeloof commented 10 months ago

my bad, I missread your answer - reopened

mathroc commented 4 months ago

I can confirm the issue. docker compose pull uses the proxy, but docker compose build --pull doesn't

Docker Compose version v2.26.1