moby / buildkit

concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit
https://github.com/moby/moby/issues/34227
Apache License 2.0
8.21k stars 1.16k forks source link

docs: proposal to raise awareness about an unexpected behavior of COPY --link #4964

Open IceCodeNew opened 5 months ago

IceCodeNew commented 5 months ago

Situation:

Amended the COPY instruction with --link to make better usage of the build cache, and unexpectedly introduced a file permission problem.

I copied the directory /home/nonroot/ of a builder stage to the rootless final stage. And found out that the newly deployed image could not work properly because the ownership of /home/nonroot/ became root:root

How to reproduce:

# syntax=docker/dockerfile:1

FROM gcr.io/distroless/static:debug-nonroot AS build
WORKDIR /home/nonroot/
SHELL ["/busybox/sh", "-eo", "pipefail", "-c"]
RUN touch test \
    && ls -halF

FROM gcr.io/distroless/static:debug-nonroot
COPY --link --from=build /home/nonroot/ /emptydir/home/nonroot/
WORKDIR /emptydir/home/nonroot/
SHELL ["/busybox/sh", "-eo", "pipefail", "-c"]
RUN ls -halF
 docker build --no-cache --progress plain -t copy-link -f copy-link.dockerfile .
#0 building with "orbstack" instance using docker driver

#1 [internal] load build definition from copy-link.dockerfile
#1 transferring dockerfile: 436B done
#1 DONE 0.0s

#2 resolve image config for docker.io/docker/dockerfile:1
#2 DONE 0.0s

#3 docker-image://docker.io/docker/dockerfile:1
#3 CACHED

#4 [internal] load metadata for gcr.io/distroless/static:debug-nonroot
#4 DONE 0.0s

#5 [internal] load .dockerignore
#5 transferring context: 2B done
#5 DONE 0.0s

#6 [build 1/3] FROM gcr.io/distroless/static:debug-nonroot
#6 CACHED

#7 [build 2/3] WORKDIR /home/nonroot/
#7 CACHED

#8 [build 3/3] RUN touch test     && ls -halF
#8 0.065 total 0      
#8 0.065 drwx------    1 nonroot  nonroot        8 May 29 15:01 ./
#8 0.065 drwxr-xr-x    1 nonroot  nonroot       14 Jan  1  1970 ../
#8 0.065 -rw-r--r--    1 nonroot  nonroot        0 May 29 15:01 test
#8 DONE 0.1s

#9 [stage-1 2/4] COPY --link --from=build /home/nonroot/ /emptydir/home/nonroot/
#9 DONE 0.0s

#10 [stage-1 3/4] WORKDIR /emptydir/home/nonroot/
#10 DONE 0.0s

#11 [stage-1 4/4] RUN ls -halF
#11 0.065 total 0      
#11 0.065 drwxr-xr-x    1 root     root           8 May 29 15:01 ./
#11 0.065 drwxr-xr-x    1 root     root          14 May 29 15:01 ../
#11 0.065 -rw-r--r--    1 nonroot  nonroot        0 May 29 15:01 test
#11 DONE 0.1s

#12 exporting to image
#12 exporting layers 0.0s done
#12 writing image sha256:e0c07fc57c86d911f7f1eed7ab2b189e87e5f1fb14018d0267d1bbe83b569e0c done
#12 naming to docker.io/library/copy-link done
#12 DONE 0.0s

Discussion:

Please note that the ownership of /home/nonroot/ is #8 0.065 drwx------ 1 nonroot nonroot 8 May 29 15:01 ./

while the ownership of /emptydir/home/nonroot/ in the final stage is #11 0.065 drwxr-xr-x 1 root root 8 May 29 15:01 ./

Since the COPY --link basically works like

FROM scratch AS linker
COPY --from=builder /home/nonroot/ /home/nonroot/

FROM last-stage-base
COPY --from=linker / /

I guess it makes sense for the unexpected ownership change of /home/nonroot/

I am not proposing to change the quirk of COPY --link, instead I think it is important for people to notice such a discrepancy of behavior among the instructions COPY & COPY --link.

Solution:

  1. also set the --chown arg while issuing COPY --link, with the limitation of only numeric UID/GID could work ( #2987 ). This idea may not be preferred if the UID/GID of the previous stage is prone to changes.
  2. put the directory you want to copy under an empty dir and COPY --link that empty dir instead.
  3. changes to the code so that this quirk is fixed.

I am willing to contribute to the docs should this proposal be accepted. Before that, I would like to know if changes to the code or other actions are preferred.

IceCodeNew commented 3 months ago

Hi @tonistiigi ,

I apologize for the interruption, but I wanted to bring to your attention that this proposal seems to have stalled due to the lack of an assignee.

Would you be able to refer someone who could provide direction on which solution we should proceed with?

Thank you for your time and assistance.

polarathene commented 3 months ago

UPDATE: I've covered this concern with a few other related issues over at: https://github.com/docker/docs/issues/20660

I won't have time to contribute proper docs improvements myself, but if you do tackle that please update the linked issue with your related PR for traceability :)


the newly deployed image could not work properly because the ownership of /home/nonroot/ became root:root

This --link quirk only seems to affect docker-container driver. I agree it should be documented for clarity, in the meantime I've done so unofficially here.

You can avoid this problem by using the docker driver (docker buildx build --builder default ...), or by adjusting your Dockerfile to use COPY --link --chmod=null workaround.

NOTE: I'm not 100% sure if --link functionality is still applied, or if either solution triggers --link=false. Is there an easy way to verify locally?


# syntax=docker.io/docker/dockerfile:1

# `src` has parent dirs with permissions and ownership: `777 100:100`
FROM alpine AS src
WORKDIR /foo/bar/baz
RUN <<HEREDOC
  touch a b c
  chown -R 100:100 /foo
  chmod -R 777 /foo
HEREDOC

# `dest` has parent dirs with permissions and ownership: `770 200:200`
FROM alpine AS dest
RUN <<HEREDOC
  mkdir -p /foo/bar/baz
  chown -R 200:200 /foo
  chmod -R 770 /foo
HEREDOC

# Workaround to prevent modifying ownership and permissions of an existing COPY target path segments:
# NOTE: `--chmod` can use any invalid value to prevent the caveat of `--link` + `docker-container` builder driver.
FROM dest AS link-workaround-fix
COPY --link --chmod=null --from=src /foo/bar/baz /foo/bar/baz
# `eza` is a fancier `ls` command. Used here for better visualizing changes:
RUN apk --no-cache add eza
CMD eza -lanhog --tree --no-time --no-filesize --no-permissions /foo
docker buildx build --load --tag bug-copy-link .
docker run --rm -it bug-copy-link

You'll get this output from the eza CMD:

image

Whereas without the --chmod workaround, as experienced the target path ownership and permissions are modified to 755 0:0:

image


Related concerns

NOTE:

That last one if important to you may be problematic due to sibling content also getting copied over, you could use --parents or --exclude to workaround that:

# syntax=docker/dockerfile:1.7-labs

# ...

# Preserve `baz/` ownership/permissions from src via --parents `./` syntax:
COPY --link --chmod=null --from=src --parents /foo/bar/./baz /foo/bar

# Preserve `bar/baz/` ownership/permissions from src via --exclude filter on src `/foo` dir
# NOTE: There is no negative match pattern like `!` for a substring supported, thus you must explicitly ignore all unwanted dirs
COPY --link --chmod=null --from=src --exclude=/foo/dont-copy-me /foo /foo