search

moby / buildkit

concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit
https://github.com/moby/moby/issues/34227
Apache License 2.0
8.25k stars 1.17k forks source link

Buildkit unable to build image with chmod -R 0 /etc #3564

Open gbhushan86 opened 1 year ago

gbhushan86 commented 1 year ago

Description

Hi All, IM trying to build image in Docker Desktop v20.10.17 using Buildkit=true.

Below is the Docker File

FROM dockerhub.rnd.amadeus.net:5002/uf-acs/jboss_standard:5.7.3.9.1
ENV DT_AGENT_ROOT=/opt/dynatrace-agent
ENV dt_agent_installer="dynatrace-agent-7.0.0.2469-unix.jar"
RUN curl -o /opt/${dt_agent_installer} https://repository.rnd.amadeus.net/acs-project/praxis/dynatrace/${dt_agent_installer} && \
 java -jar /opt/${dt_agent_installer} -t ${DT_AGENT_ROOT} -b 64 && \
 rm /opt/${dt_agent_installer}
USER root

ENV JBOSS_DEPLOY_DIR=/deployments

ADD ./root-server /
RUN sed -i 's/##CONSOLE-FORMATTER##/COLOR-PATTERN/' /opt/eap/standalone/configuration/logging.properties

RUN mkdir -p /logs/ard-cryptic-app/ard-cryptic-domain/app /logs/ard-cryptic-app/ard-cryptic-domain/hosttrace /logs/ard-cryptic-app/ard-cryptic-domain/record /logs/ard-cryptic-app/ard-cryptic-domain/replay /logs/ard-cryptic-app/ard-cryptic-domain/statistic

RUN yum install -y tar rsync && \
    yum install libaio -y
RUN chmod +x /opt/eap/bin/*.sh && \
    unlink /etc/localtime && ln -s /usr/share/zoneinfo/UTC /etc/localtime && \
    echo "unset JAVA_TOOL_OPTIONS" >> /home/jboss/.bashrc && \
    rm -rf ${JBOSS_DEPLOY_DIR}/activemq-rar.rar && \    
    rm -rf ${JBOSS_HOME}/standalone/configuration/standalone_xml_history/* && \
    chmod 777 /opt/eap/standalone/configuration/*.properties
RUN sed -i 's/jolokia.jar=port=8778,protocol=https/jolokia.jar=port=8778,protocol=http/g' /opt/eap/bin/standalone.conf
RUN chmod +x /opt/eap/bin/openshift-launch.sh
RUN chmod +x /opt/eap/bin/startup-script-wrapper.sh

RUN chgrp -R 0 /opt/eap/ && \
    chmod -R g=u /opt/eap/

RUN **chgrp -R 0 /etc** && \
    chmod -R g=u /etc

RUN chgrp -R 0 /opt/eap/standalone/deployments && \
    chmod -R g=u /opt/eap/standalone/deployments

RUN chgrp -R 0 ${JBOSS_DEPLOY_DIR} && \
    chmod -R g=u ${JBOSS_DEPLOY_DIR}

While running the command while enabling buildkit Im receiving the following error.

    [+] Building 47.5s (16/18)
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 2.19kB                                     0.0s
 => [internal] load .dockerignore                                          0.1s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for dockerhub.rnd.amadeus.net:5002/uf-acs/jb  0.0s
 => [ 1/14] FROM dockerhub.rnd.amadeus.net:5002/uf-acs/jboss_standard:5.7  0.0s
 => [internal] load build context                                         21.3s
 => => transferring context: 148.07MB                                     21.3s
 => CACHED [ 2/14] RUN curl -o /opt/dynatrace-agent-7.0.0.2469-unix.jar h  0.0s
 => [ 3/14] ADD ./root-server /                                            1.4s
 => [ 4/14] RUN sed -i 's/##CONSOLE-FORMATTER##/COLOR-PATTERN/' /opt/eap/  0.5s
 => [ 5/14] RUN mkdir -p /logs/ard-cryptic-app/ard-cryptic-domain/app /lo  0.6s
 => [ 6/14] RUN yum install -y tar rsync &&     yum install libaio -y     13.9s
 => [ 7/14] RUN chmod +x /opt/eap/bin/*.sh &&     unlink /etc/localtime &  0.6s
 => [ 8/14] RUN sed -i 's/jolokia.jar=port=8778,protocol=https/jolokia.ja  0.5s
 => [ 9/14] RUN chmod +x /opt/eap/bin/openshift-launch.sh                  0.5s
 => [10/14] RUN chmod +x /opt/eap/bin/startup-script-wrapper.sh            0.6s
 => [11/14] RUN chgrp -R 0 /opt/eap/ &&     chmod -R g=u /opt/eap/         6.2s
 => ERROR [12/14] RUN chgrp -R 0 /etc &&     chmod -R g=u /etc             1.2s
------
 **> [12/14] RUN chgrp -R 0 /etc &&     chmod -R g=u /etc:**
moby/moby#16 0.679 chgrp: changing group of '/etc/resolv.conf': Read-only file system
moby/moby#16 0.682 chgrp: changing group of '/etc/hosts': Read-only file system
------
executor failed running [/bin/sh -c chgrp -R 0 /etc &&     chmod -R g=u /etc]: exit code: 1
docker tag dockerhub.rnd.amadeus.net:5000/aek-acs/ard-cryptic-domain:1.0
Im using below settings in the Docker desktop:
"features": {
 "buildkit": true
},

While keeping buildkit as false im able to build Image.But as per requirement I need buildkit to build image processing.

Reproduce

FROM dockerhub.rnd.amadeus.net:5002/uf-acs/jboss_standard:5.7.3.9.1
ENV DT_AGENT_ROOT=/opt/dynatrace-agent
ENV dt_agent_installer="dynatrace-agent-7.0.0.2469-unix.jar"
RUN curl -o /opt/${dt_agent_installer} https://repository.rnd.amadeus.net/acs-project/praxis/dynatrace/${dt_agent_installer} && \
 java -jar /opt/${dt_agent_installer} -t ${DT_AGENT_ROOT} -b 64 && \
 rm /opt/${dt_agent_installer}
USER root

ENV JBOSS_DEPLOY_DIR=/deployments

ADD ./root-server /
RUN sed -i 's/##CONSOLE-FORMATTER##/COLOR-PATTERN/' /opt/eap/standalone/configuration/logging.properties

RUN mkdir -p /logs/ard-cryptic-app/ard-cryptic-domain/app /logs/ard-cryptic-app/ard-cryptic-domain/hosttrace /logs/ard-cryptic-app/ard-cryptic-domain/record /logs/ard-cryptic-app/ard-cryptic-domain/replay /logs/ard-cryptic-app/ard-cryptic-domain/statistic

RUN yum install -y tar rsync && \
    yum install libaio -y
RUN chmod +x /opt/eap/bin/*.sh && \
    unlink /etc/localtime && ln -s /usr/share/zoneinfo/UTC /etc/localtime && \
    echo "unset JAVA_TOOL_OPTIONS" >> /home/jboss/.bashrc && \
    rm -rf ${JBOSS_DEPLOY_DIR}/activemq-rar.rar && \    
    rm -rf ${JBOSS_HOME}/standalone/configuration/standalone_xml_history/* && \
    chmod 777 /opt/eap/standalone/configuration/*.properties
RUN sed -i 's/jolokia.jar=port=8778,protocol=https/jolokia.jar=port=8778,protocol=http/g' /opt/eap/bin/standalone.conf
RUN chmod +x /opt/eap/bin/openshift-launch.sh
RUN chmod +x /opt/eap/bin/startup-script-wrapper.sh

RUN chgrp -R 0 /opt/eap/ && \
    chmod -R g=u /opt/eap/

RUN chgrp -R 0 /etc && \
    chmod -R g=u /etc

RUN chgrp -R 0 /opt/eap/standalone/deployments && \
    chmod -R g=u /opt/eap/standalone/deployments

RUN chgrp -R 0 ${JBOSS_DEPLOY_DIR} && \
    chmod -R g=u ${JBOSS_DEPLOY_DIR}

# If you want to see all console logs of startup script inside a local log file you can activate this entry point
# to be used in developpment phase if you are not able to understand JBoss fails to start
# useful when migrating to JBoss version or if we adapt standalone-openshift.xml
# ENTRYPOINT ["/opt/eap/bin/startup-script-wrapper.sh"]

LABEL ard-cryptic-domain ${project.version}

Expected behavior

Buildkit should build image without any errors

docker version

Client:
 Cloud integration: v1.0.24
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:09:02 2022
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.10.1 (82475)
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:01:23 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.6
  GitCommit:        10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc:
  Version:          1.1.2
  GitCommit:        v1.1.2-0-ga916309
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.2)
  compose: Docker Compose (Docker Inc., v2.6.1)
  extension: Manages Docker extensions (Docker Inc., v0.2.7)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 192
  Running: 52
  Paused: 0
  Stopped: 140
 Images: 92
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.16.3-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.771GiB
 Name: docker-desktop
 ID: YLF7:EKSS:MT7M:U7XU:2466:7XB5:N66I:N5SL:TQNL:MZJL:KXW3:VXWL
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Registry Mirrors:

 Live Restore Enabled: false

Additional Info

No response

thaJeztah commented 1 year ago

So the /etc/resolv.conf, /etc/hostname and /etc/hosts files (with the default bridge network at least) are managed files, and mounted into the container, so it's worth noting that;

That said, it looks like BuildKit mounts those read-only (whereas the legacy builder does allow them to be modified);

Without Buildkit:

DOCKER_BUILDKIT=0 docker build --no-cache -<<'EOF'
FROM alpine
RUN mount | grep /etc
EOF

Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM alpine
 ---> 042a816809aa
Step 2/2 : RUN mount | grep /etc
 ---> Running in 4439f724c5b4
/dev/vda1 on /etc/resolv.conf type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /etc/hostname type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /etc/hosts type ext4 (rw,relatime,data=ordered)

With BuildKit:

DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -<<'EOF'
FROM alpine
RUN mount | grep /etc
EOF

moby/moby#5 [2/2] RUN mount | grep /etc
moby/moby#5 sha256:6a771f791ee459e50e92baa65f3ca068473cf1f9191b6ff6a5f488b2378ba88e
moby/moby#5 0.194 /dev/vda1 on /etc/resolv.conf type ext4 (ro,nosuid,nodev,noexec,relatime)
moby/moby#5 0.194 /dev/vda1 on /etc/hosts type ext4 (ro,nosuid,nodev,noexec,relatime)
moby/moby#5 DONE 0.2s
thaJeztah commented 1 year ago

Let me transfer this to the BuildKit repository, as changing this would require changes in that repository (but it's possible this was by design).

gbhushan86 commented 1 year ago

So the /etc/resolv.conf, /etc/hostname and /etc/hosts files (with the default bridge network at least) are managed files, and mounted into the container, so it's worth noting that;

  • changes in permissions of those files won't persist in the image (as they're bind-mounted when a container is started from them)
  • I do notice that you're recursively changing all files in /etc; was there a specific reason for that? (there's quite some files under that path, changing permissions means those files will be duplicated into a new layer)

That said, it looks like BuildKit mounts those read-only (whereas the legacy builder does allow them to be modified);

Without Buildkit:

DOCKER_BUILDKIT=0 docker build --no-cache -<<'EOF'
FROM alpine
RUN mount | grep /etc
EOF

Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM alpine
 ---> 042a816809aa
Step 2/2 : RUN mount | grep /etc
 ---> Running in 4439f724c5b4
/dev/vda1 on /etc/resolv.conf type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /etc/hostname type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /etc/hosts type ext4 (rw,relatime,data=ordered)

With BuildKit:

DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain -<<'EOF'
FROM alpine
RUN mount | grep /etc
EOF

moby/moby#5 [2/2] RUN mount | grep /etc
moby/moby#5 sha256:6a771f791ee459e50e92baa65f3ca068473cf1f9191b6ff6a5f488b2378ba88e
moby/moby#5 0.194 /dev/vda1 on /etc/resolv.conf type ext4 (ro,nosuid,nodev,noexec,relatime)
moby/moby#5 0.194 /dev/vda1 on /etc/hosts type ext4 (ro,nosuid,nodev,noexec,relatime)
moby/moby#5 DONE 0.2s

Since its a container we dont want other users to touch or tweak any files inside /etc so why..