[BUG] Secret not being read from Environment

rbargerhuff commented 1 month ago

Description

Secrets defined within the secrets section of compose.yml are not being read byv2.28.1 docker-compose.

secrets:
  gitlab_username:
    environment: "GITLAB_USERNAME"
  gitlab_access_token:
    environment: "GITLAB_ACCESS_TOKEN"
  mysql_root_password:
    environment: "MYSQL_ROOT_PASSWORD"

This is defined in accordance with current docker documentation. Please see Example 2 under the following documentation:

https://docs.docker.com/compose/compose-file/09-secrets/

None of our containers can build using v2.28.1.

Downgrading docker-compose to v2.27.1 fixes the issue.

Steps To Reproduce

I do not have easy steps that I can provide to produce this issue. If further information is needed, I can provide as necessary.

Compose Version

Docker Compose version v2.28.1

Docker Environment

root@rodev03 /home/bargerhuff $ docker info
Client: Docker Engine - Community
 Version:    27.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.15.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.28.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 34
  Running: 32
  Paused: 0
  Stopped: 2
 Images: 65
 Server Version: 27.0.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.14.0-427.24.1.el9_4.x86_64
 Operating System: AlmaLinux 9.4 (Seafoam Ocelot)
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 15.37GiB
 Name: rodev03.rowan.edu
 ID: acc1cdd1-9298-40e7-a980-3a30584eb26e
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

Error received:

[+] Building 0.9s (10/77) docker:default => CACHED [portal-dev internal] load git source https://maio:glpat-XXXXXN@gitlab.site.edu/repos/app/portal.git#main 0.2s => [portal-dev internal] load metadata for docker.io/library/php:8.3-apache 0.1s => [portal-dev internal] load metadata for docker.io/bitnami/git:2.44.0 0.1s => CACHED [portal-dev base 46/50] ADD https://github.com/just-containers/s6-overlay/releases/download/v3.2.0.0/s6-overlay-x86_64.tar.xz /tmp 0.1s => CACHED [portal-dev base 44/50] ADD https://github.com/just-containers/s6-overlay/releases/download/v3.2.0.0/s6-overlay-noarch.tar.xz /tmp 0.1s => CACHED [portal-dev mod_auth_cas 1/6] FROM docker.io/library/php:8.3-apache@sha256:cb992558faa44e6ed0800740f8fd3dac5ba4ac26644c46f8dc53e97f70095838 0.0s => CACHED [portal-dev git 1/6] FROM docker.io/bitnami/git:2.44.0@sha256:3e81903161a5cf24d79fabefe9d6268664b81ee62092c01a402949237f7d53a9 0.0s => [portal-dev base 2/50] RUN ln -sf /usr/share/zoneinfo/US/Eastern /etc/localtime 0.2s => ERROR [portal-dev git 2/6] RUN --mount=type=secret,id=gitlab_username --mount=type=secret,id=gitlab_accesstoken git clone -b main https://$(</run/secrets/gitlab 0.4s => CANCELED [portal-dev base 3/50] RUN apt-get update

jhrotko commented 1 month ago

looking at your log => ERROR [portal-dev git 2/6] RUN --mount=type=secret,id=gitlab_username --mount=type=secret,id=gitlab_access_token git clone -b main https://$(</run/secrets/gitlab_ 0.4s It seems when you are building your image, there is an error in this line. It is a bit hard to help with such a minimal compose example. I assume you have a build section for your service did you add your secrets in your build?

rbargerhuff commented 1 month ago

looking at your log => ERROR [portal-dev git 2/6] RUN --mount=type=secret,id=gitlab_username --mount=type=secret,id=gitlab_access_token git clone -b main https://$(</run/secrets/gitlab_ 0.4s It seems when you are building your image, there is an error in this line. It is a bit hard to help with such a minimal compose example. I assume you have a build section for your service did you add your secrets in your build?

That is correct because this error is being generated due to the fact that the secret cannot be referenced /run/secrets/gitlab_ 0.4s is empty.

This same line does not error out in v2.27.1

jhrotko commented 1 month ago

I am wondering if you add target=/run/secrets/gitlab_username to RUN --mount=type=secret,id=gitlab_username,target=/run/secrets/gitlab_username ... if it helps

rkarp commented 3 weeks ago

I am wondering if you add target=/run/secrets/gitlab_username to RUN --mount=type=secret,id=gitlab_username,target=/run/secrets/gitlab_username ... if it helps

I tried this, the secret file at /run/secrets/xyz is still empty (size 0).

docker / compose