Open ldanielw1 opened 1 month ago
compose does not run the build on it's own but just creates a build configuration for buildkit to build image(s) Can you reproduce this behaviour with Docker Desktop? Otherwise this is a Colima issue you should report to the authors
that's the crazy thing - I can't reproduce this behavior with Docker Desktop. But I also can't reproduce this issue with docker
on colima. So something is different between how docker-compose
is creating layers, vs. how docker
is doing it. So - wouldn't that be an issue of how docker-compose
differs from docker
? I could be naive, though, so lmk if there's something I'm missing (I have already raised this issue with them, but am still waiting for a response)
that's indeed probably an issue with compose configuring build execution (compose just setup buildx to run the build, it doesn't create layers on its own). We will welcome any PR to offer a fix so this works well on Colima, but you will easily understand that, as a Docker employee, I can't spend time debugging an issue that only applies to a competing product :)
Have you tried modifying the BuildKit settings directly if possible, or forcing Docker Compose to use a different build method or configuration to see if that affects the caching behavior?
Also could you compare the build context from Docker vs. Docker Compose? Sometimes there might be differences in how the context is packaged and sent to the builder which can affect caching.
@ndeloof - of course, I appreciate your pointing me in the right direction, though.
@neelp03 - well, i'm currently running builds with BuildKit turned off (with a DOCKER_BUILDKIT=0
). And strangely enough, now that you're asking, the problem goes away if I set it back to 1. However, I kind of need buildkit to be off for the time-being (for debug purposes).
It guess there is something about how compose runs when on colima AND when buildkit is off, where it is creating layers uniquely from build to build such that it always has a cache-miss before building layers...
Hi! I'm also running into this problem, but not on Colima. I use Docker Desktop and when I try to rebuild an existing image with docker build
caching works and it's super fast, but when I try to do so with docker compose build
caching seems to not work as well. What I mean is, when rebuilding with docker compose
, the first few commands will reuse the cache but after that, the build progresses as if it were building from scratch even though nothing had changed.
Here are my docker env details
Client:
Version: 26.1.4
API version: 1.45
Go version: go1.21.11
Git commit: 5650f9b
Built: Wed Jun 5 11:26:02 2024
OS/Arch: darwin/arm64
Context: desktop-linux
Server: Docker Desktop 4.31.0 (153195)
Engine:
Version: 26.1.4
API version: 1.45 (minimum version 1.24)
Go version: go1.21.11
Git commit: de5c9cf
Built: Wed Jun 5 11:29:12 2024
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.6.33
GitCommit: d2d58213f83a351ca8f528a95fbd145f5654e957
runc:
Version: 1.1.12
GitCommit: v1.1.12-0-g51d5e94
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Docker Compose version v2.27.1-desktop.1
I also have both DOCKER_BUILDKIT
and COMPOSE_DOCKER_CLI_BUILD
set to 1
Description
I'm on an M1 mac with colima. When I try to run
docker build .
in my repo, it caches layers - but when I rundocker-compose build
, it doesn't. AND it deletes the layers cached when I had rundocker build .
I also noticed that every time I rundocker-compose build
, the number of layers is increasing in my colima machine's/var/lib/docker/overlay2
. So I'm guessing it just can't find the cached layers and is continuing to create new layers in that folder?I'm curious why this might be the case and how I can get my docker-compose builds to start caching layers again?
Steps To Reproduce
No response
Compose Version
Docker Environment
Anything else?
No response