Open neilyio opened 3 years ago
I realize this is probably because of the parallelism that is part of the compose-cli
project, but this seems to be a case where that parallelism is getting in the way.
It seems to me this indeed only works "by chance" as docker-compose build
run sequentially, building first image with a tag, then using it as base image for the next service.
I'm not sure if/how we can support such a use-case, would need to check how buildkit can handle this.
I'm facing this issue too. Is it possible to analyze the build dependency by marking possibly used as base image against the images which specify both image
and build
? Do we need a new option to specify build dependency in the compose config?
Having this issue as well. I tried defining depends_on:
on the other services but still didn't work. Went back to use docker-compose
(2nd time I had to bail on using the new command)
Do we need a new option to specify build dependency in the compose config?
IMO this is what depends_on:
was being used for previously. Docker Compose V1 was smart enough to see that an image in depends_on:
didn't exist and that when defined with local build configuration in docker-compose.yml
, to build that image first before being used.
A lot of people depend on this behaviour being the same when moving to V2.
A workaround I see is manually building the images in depends_on:
in the docker-compose.yml
first before building the rest of the images. This is a large maintenance burden though as developers would need to keep track of these image names in a static configuration/scripts at the very least. For this reason, I'm holding off on V2.
@ndeloof This issue is breaking behaviour in a lot of repos I'm seeing and should be addressed.
Same issue applies to compose v1 as long as buildkit is enabled: https://github.com/docker/compose/issues/8449
depends on https://github.com/docker/buildx/issues/447
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Still on it
This issue has been automatically marked as not stale anymore due to the recent activity.
Apparently similar to https://github.com/docker/compose/issues/8805
Are there any temporary work arounds for this problem?
Workarounds depend a bit on your exact use-case;
You can run the builds for each service manually to make sure to build the base image first (docker compose build base
), but this depends on what "builder" you use; as it won't work if you use a remote or "container" builder (such builders store build-cache, but not images).
The other workaround (this would usually be the recommended approach) is to use a multi-stage build. However, this assumes the situation as outlined in this ticket's description where both images share the same build-context.
Rewriting the example to have both services use the same Dockerfile, but a different target
(stage). The second (extended
) stage depends on the first (base
) stage, which means that building extended
will also build the base
stage.
# docker/docker-compose.yml
services:
base:
image: neilyio/base
build:
context: .
target: base
extended:
build:
context: .
target: extended
# syntax=docker/dockerfile:1
FROM scratch AS base
COPY README.md /root/README.md
FROM base AS extended
CMD cat /root/README.md
It's worth noting that;
When building only extended
, the layer(s) for base
will be built, but no image (neilyio/base
) is tagged for the base
image.
docker compose build extended
[+] Building 2.9s (9/9) FINISHED
=> [internal] load build definition from Dockerfile 0.2s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.3s
=> => transferring context: 2B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 2.0s
=> CACHED docker-image://docker.io/docker/dockerfile:1@sha256:443aab4ca21183e069e7d8b2dc68006594f40bddf1b15bbd83f5137bd93e80e2 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load .dockerignore 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 30B 0.0s
=> CACHED [base 1/1] COPY README.md /root/README.md 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:ee813e9db95b2b2d222f4cf72b4507ece85773201a26634f5066ff7d2d4bec65 0.0s
=> => naming to docker.io/library/compose-multibuild_extended 0.0s
docker image ls --filter reference=neilyio/base
REPOSITORY TAG IMAGE ID CREATED SIZE
docker compose build base
[+] Building 1.4s (9/9) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 0.8s
=> CACHED docker-image://docker.io/docker/dockerfile:1@sha256:443aab4ca21183e069e7d8b2dc68006594f40bddf1b15bbd83f5137bd93e80e2 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load .dockerignore 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 30B 0.0s
=> CACHED [base 1/1] COPY README.md /root/README.md 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:67e08e818eb01fd3b97ad637b5e99762d34c766e2bb11f2c501a716ef183de62 0.0s
=> => naming to docker.io/neilyio/base 0.0s
Generally, building extended
, then building base
should produce the same image (common layers shared between both images), but there may be some improvements to be made in compose here; if both builds would run in parallel, they're not guaranteed to produce the same layer-digests (if changes are made in between within the build-context).
docker inspect --format '{{json .RootFS.Layers}}' neilyio/base
["sha256:3b1be5298307e60029517bf38caa3f1b58121ef39d56d832b4a62dbabc56c3d3"]
docker inspect --format '{{json .RootFS.Layers}}' compose-multibuild_extended
["sha256:3b1be5298307e60029517bf38caa3f1b58121ef39d56d832b4a62dbabc56c3d3"]
@thaJeztah thank you for the detailed response. From what I've been reading online, it was my conclusion too.
My situation is tad more dependency chain problematic, where I'm building a base
and using that local image in 2 other local images (e.g. foo
and bar
) which are using FROM base
. I suppose adopting your approach would mean quite some changes to the file system structure (all images are built from root/base
, root/foo
, root/bar
, etc. currently). Also, when I looked at this earlier, it lead me to cache-from
and target
in docker-compose.yml
and I was really hoping to avoid all that.
Any idea on ETA / priority for this issue?
Thanks again.
@curlybeast Recent docker compose release will build images with respect to the depends_on
directive, so that you can declare the base
image to be built first, then the other service images.
Not sure if this valid or not, but I have same issue and I solve it by using this 2 commands :
export DOCKER_BUILDKIT=0
export COMPOSE_DOCKER_CLI_BUILD=0
After i run that I re-run docker-compose up/build and its solved.
@curlybeast Recent docker compose release will build images with respect to the
depends_on
directive, so that you can declare thebase
image to be built first, then the other service images.
Thanks for letting me know about this. Which version is this released in?
for me this is fixed when i start use docker compose
instead of docker-compose
. because docker-compose
outdated.
for me this is fixed when i start use
docker compose
instead ofdocker-compose
. becausedocker-compose
outdated.
Just be careful you don't have another version in your PATH
. I just realised I had an old docker-compose
in /usr/local/bin
from back in the days when it wasn't installed as standard with docker
packages. This is why I couldn't see any of the new features at first :roll_eyes: (like docker compose pull --ignore-pull-failures
)
To better cover this scenario, it seems to be we should define a new depends_on condition dedicated to build requirements, i.e.
services:
base:
build: .
extened:
build:
context: extended
depends_on:
base:
condition: image_built
I'll experiment with this approach and prepare a proposal on https://github.com/compose-spec/compose-spec
Any updates on this issue? It seems like docker compose v2.18.1 doesn't respect depends_on for the build when build kit is enabled.
@pebo can you provide a simple example to reproduce this issue?
Docker Compose run builds in depends_on
order to address this need (while dependency might not be required at runtime, so my comment)
@ndeloof It seems like the problem arises when the dependent service is using the image
property rather than a build
section.
Dockerfile
FROM alpine as base
RUN echo 'foo' > /foo.txt
RUN echo 'sleeping for 10s' && sleep 10
FROM base-image as sub
RUN echo 'bar' > /bar.txt && cat /foo.txt >> /bar.txt
docker-compose.ml
version: "3"
services:
base:
image: base-image
build:
context: .
dockerfile: ./Dockerfile
target: base
deploy:
replicas: 0
sub-with-build:
build:
context: .
dockerfile: ./Dockerfile
target: sub
command: echo 'sub-with-build' && cat /bar.txt
depends_on:
- base
sub-with-image:
image: base-image
command: echo 'sub-with-image' && cat /bar.txt
depends_on:
- base
Error
docker compose up
[+] Running 2/2
✘ sub-with-image Error
The service sub-with-build
waits for base-image
to be built and then runs as expected but sub-with-image
fails. Is this the expected outcome?
@pebo Can confirm this. I had one base image and it was working. Then tried adding a second "stage" (since a few images share part of the features) and suddenly everything fell apart.
Some folks have noted that depends_on
controls the order of the build. I am not so sure. Official docs says that depends_on
is used to control the startup and shutdown sequence of the containers. So it's a runtime configuration, not a build configuration. See https://docs.docker.com/compose/compose-file/05-services/#depends_on
Apologies if my understanding is wrong.
Some folks have noted that
depends_on
controls the order of the build. I am not so sure. Official docs says thatdepends_on
is used to control the startup and shutdown sequence of the containers. So it's a runtime configuration, not a build configuration. See https://docs.docker.com/compose/compose-file/05-services/#depends_on Apologies if my understanding is wrong.
Even though undocumented, depend_on
used to control the build order as well (haven't tested it recently). And unfortunately it's the only way to control the build order without using third-party tools. For some reason, docker-compose maintainers decided to exclude build ordering from the docker-compose feature set, despite most of the code being clearly present and working.
Be careful when using buildkit features. Buildkit uses dockerized build-instances and therefore this instances have some isolation against host system (and host docker). I did not realize at first, what consequences that can have.
If you have two docker-compose services with services that depend on base-image build by another service this can break if you are using separate Dockerfiles!
https://github.com/moby/buildkit/issues/4162#issuecomment-1686304524
export DOCKER_BUILDKIT=0
worked for me on WSL2. Looks like buildkit
usage is recent? I would expect it to work as previously though.
Description
I'm getting an error from
docker compose build
with a setup that is consistently successful withdocker-compose build
.I have a
docker-compose.yml
file with two services:base
andextended
.base.image
gives a name to the image built bybase
, and I'd like to use that name as theFROM
image in the Dockerfile for theextended
service.This works well with
docker-compose build
. It does not work withdocker compose build
.Steps to reproduce the issue:
Re-create my
docker/
folder with these three files, plus an emptyREADME.md
.cd
intodocker/
.docker-compose build
, expect a successful run.docker compose build
, expect a failure.Describe the results you received:
docker compose build
produces:Describe the results you expected:
I expected
docker compose build
to find the locally-builtneilyio/base
image, instead it seems to try and load fromdocker.io/neilyio/base:latest
. I expecteddocker compose build
to have the same behaviour asdocker-compose build
, which successfully found the local image.Additional information you deem important (e.g. issue happens only occasionally):
This can be a little tricky to reproduce because of Docker's caching.
docker compose build
will work fine ifneilyio/base
is already built.docker compose build
will successfully find the local image, so it can give the impression that it's working. My step 3 above, clearing the cache, is important to accurately reproduce this. I found I needed to do both asystem prune
andimage rm
for this.docker-compose build
works every time, whether or notneilyio/base
has been built before.Output of
docker version
:Output of
docker context show
:Output of
docker info
:Additional environment details (AWS ECS, Azure ACI, local, etc.):
Local run on M1 Macbook Air.