Open Strawl opened 9 months ago
Thanks for taking the time to create this issue @Strawl, I'm happy to try and better describe how GC works with BuildKit. (Agree that the current description could be improved!)
I can create list of all these inconsistencies that i have found and which i do not understand.
That would be very helpful, thanks.
Also cc'ing in @tonistiigi and @crazy-max for visibility/comments.
Thanks @dvdksn for taking up this issue!
Yesterday, I actually did manage to configure GC with the default policy. A few days before trying to configure GC, I also had enabled "live-restore" (and had correctly done a normal daemon restart). The documentation about live-restore states:
On Linux, you can avoid a restart (and avoid any downtime for your containers) by reloading the Docker daemon. If you use systemd, then use the command systemctl reload docker
This is actually a lot where my confusion stemmed from... a simple systemctl reload docker
was not enough in my case to apply GC configuration for the default docker daemon. I did indeed have to do a systemctl restart docker
to get the GC configuration applied. In the end, the containers were not restarted. What also helped was to actually know about the command docker buildx/builder inspect, which, among other things, shows the GC policies configured for a given builder.
On my own pc, I also managed to reproduce this behavior:
systemctl restart docker
systemctl reload docker
docker buildx inspect
--> no policies addedsystemctl restart docker
docker buildx inspect
--> policies visibleSo to summarize this comment:
Sometime next week, I'll create the (actual) list I mentioned in the description.
Related to GC with cache mounts, let me know if this should be raised as a separate issue.
https://docs.docker.com/engine/reference/builder/#run---mounttypecache
Contents of the cache directories persists between builder invocations without invalidating the instruction cache. Cache mounts should only be used for better performance. Your build should work with any contents of the cache directory as another build may overwrite the files or GC may clean it if more storage space is needed.
The latter could perhaps clarify that the GC clean concern should not happen mid-build? (presumably?)
It's a tad vague on the risk of a build starting that has one or more RUN
with that cache mount used, if GC would discard it. Perhaps it does if it's not actively mounted (while handling a separate instruction before RUN
), but I'd assume it's not clearing the mount contents when they're actively used?
As another build can share the cache mount unless explicitly setting a different sharing
option to avoid unexpected deletions, following up with the GC concern may imply that could also be cleared while in active use.
There hasn't been any activity on this issue for a long time.
If the problem is still relevant, mark the issue as fresh with a /remove-lifecycle stale
comment.
If not, this issue will be closed in 14 days. This helps our maintainers focus on the active issues.
Prevent issues from auto-closing with a /lifecycle frozen
comment.
/lifecycle stale
Is this a docs issue?
Type of issue
I can't find what I'm looking for
Description
In the docs the following is stated:
But no where in the docs is it stated, how often this period is and I would have to assume it is not configurable.
Furthermore, what is
"defaultKeepStorage": "10GB"
referring to? Is this the threshold at which the pruning will start?I read also a documentation on the topic that was never merged in: https://github.com/docker/cli/pull/2864 This proposed new documentation attempted to explain how these policies work, which i find goes into the right direction, but it still left me confused. I can create list of all these inconsistencies that i have found and which i do not understand.
Location
https://docs.docker.com/build/cache/garbage-collection/
Suggestion
No response