search

docker / for-mac

Bug reports for Docker Desktop for Mac
https://www.docker.com/products/docker#/mac
2.44k stars 117 forks source link

Docker BuildKit skips stages #6764

Open Tzvetelin88 opened 1 year ago

Tzvetelin88 commented 1 year ago

Expected behavior

Build all stages with the same "--from="

Actual behavior

Only last stage is builded.

Information

Output of /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check

Starting diagnostics

[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0031: does the Docker API work?
[PASS] DD0004: is the Docker engine running?
[PASS] DD0011: are the LinuxKit services running?
[FAIL] DD0016: is the LinuxKit VM running? vm is not running: vm has not started
[PASS] DD0001: is the application running?
[PASS] DD0018: does the host support virtualization?
[FAIL] DD0017: can a VM be started? vm has not started: vm has not started
[PASS] DD0015: are the binary symlinks installed?
[PASS] DD0003: is the Docker CLI working?
[PASS] DD0013: is the $PATH ok?
[PASS] DD0007: is the backend responding?
[PASS] DD0014: are the backend processes running?
[PASS] DD0008: is the native API responding?
[PASS] DD0009: is the vpnkit API responding?
[PASS] DD0010: is the Docker API proxy responding?
[FAIL] DD0012: is the VM networking working? network checks failed: failed to ping host: exit status 1
[2023-03-13T18:11:04.961750000Z][com.docker.diagnose][I] ipc.NewClient: af0ce114-diagnose-network -> diagnosticd.sock diagnosticsd
[common/pkg/diagkit/gather/diagnose.runIsVMNetworkingOK()
[   common/pkg/diagkit/gather/diagnose/network.go:34 +0xd9
[common/pkg/diagkit/gather/diagnose.(*test).GetResult(0x4d6fb40)
[   common/pkg/diagkit/gather/diagnose/test.go:46 +0x43
[common/pkg/diagkit/gather/diagnose.Run.func1(0x4d6fb40)
[   common/pkg/diagkit/gather/diagnose/run.go:17 +0x5a
[common/pkg/diagkit/gather/diagnose.walkOnce.func1(0x2?, 0x4d6fb40)
[   common/pkg/diagkit/gather/diagnose/run.go:140 +0x77
[common/pkg/diagkit/gather/diagnose.walkDepthFirst(0x1, 0x4d6fb40, 0xc000379720)
[   common/pkg/diagkit/gather/diagnose/run.go:146 +0x36
[common/pkg/diagkit/gather/diagnose.walkDepthFirst(0x0, 0xc00061f740?, 0xc000379720)
[   common/pkg/diagkit/gather/diagnose/run.go:149 +0x73
[common/pkg/diagkit/gather/diagnose.walkOnce(0x470a100?, 0xc00061f888)
[   common/pkg/diagkit/gather/diagnose/run.go:135 +0xcc
[common/pkg/diagkit/gather/diagnose.Run(0x4d6fcc0, 0xc00061fa80?, {0xc00061fb18, 0x1, 0x1})
[   common/pkg/diagkit/gather/diagnose/run.go:16 +0x1d4
[main.checkCmd({0xc0001ac010?, 0x6?, 0x4?}, {0x0, 0x0})
[   common/cmd/com.docker.diagnose/main.go:133 +0x105
[main.main()
[   common/cmd/com.docker.diagnose/main.go:99 +0x2a7
[2023-03-13T18:11:04.962903000Z][com.docker.diagnose][I] (ba413add) af0ce114-diagnose-network C->S diagnosticsd POST /check-network-connectivity: {"ips":["192.168.100.10","10.93.145.154"]}
[2023-03-13T18:11:05.498224000Z][com.docker.diagnose][W] (ba413add) af0ce114-diagnose-network C<-S 0f2c0bcb-diagnosticsd POST /check-network-connectivity (535.484975ms): failed to ping host: exit status 1

[PASS] DD0032: do Docker networks overlap with host IPs?
[SKIP] DD0030: is the image access management authorized?
[PASS] DD0019: is the com.docker.vmnetd process responding?
[PASS] DD0033: does the host have Internet access?

Please investigate the following 1 issue:

1 : The test: can a VM be started?
    Failed with: vm has not started: vm has not started

The Docker engine runs inside a Linux VM. Therefore we must be able to start Virtual Machines.

Steps to reproduce the behavior

docker build -t test.image:latest --file Dockerfile .

A reproducible case, Dockerfiles FTW.

FROM internal.repo/node:18.13.0 as build
....

FROM internal.repo/debian:latest as update
COPY --from=build /usr/app /usr/app
...

FROM internal.repo/debian:latest as final
COPY --from=build /usr/app /usr/app
....

Only final stage is executed, update is skipped. I read that this is normal behaviour to BuildKit, but is there a best practise way to do it?

I have tried with:

DOCKER_BUILDKIT=0 docker build -t test.image:latest --file Dockerfile .

and it worked, but it doesn't seem right to skip Docker Build Kit, as there are a lot of optimisations.

Do you know a way to perform the same "--from=" without skipping BuildKit?

I have tried to give a new FROM with new alias, but it doesn't work:

FROM internal.repo/node:18.13.0 as build
....
FROM build as intermediate
FROM internal.repo/debian:latest as update
COPY --from=intermediate /usr/app /usr/app
...

FROM internal.repo/debian:latest as final
COPY --from=build /usr/app /usr/app
....
LiangZugeng commented 1 year ago

I had the same issue before, the reason is that the final stage doesn't use any output from the update stage, and docker build uses a dependency tree to determine which stages are needed by the final stage and skips all unnecessary stages.

I solved the issue by add a dummy text file in the "unnecessary stage" and then copy the dummy file from the intermediate stage to the final stage.

FROM internal.repo/node:18.13.0 as build
....

FROM internal.repo/debian:latest as update
COPY --from=build /usr/app /usr/app
# execute other commands
RUN echo "Placeholder" > /tmp/placeholder.txt
...

FROM internal.repo/debian:latest as final
COPY --from=build /usr/app /usr/app
COPY --from=update /tmp/placeholder.txt .
....

Hope this helps.