search

moby / moby

The Moby Project - a collaborative project for the container ecosystem to assemble container-based systems
https://mobyproject.org/
Apache License 2.0
68.58k stars 18.64k forks source link

Feature: `--cache-from` with wildcards #35769

Open asottile opened 6 years ago

asottile commented 6 years ago

Wanted to get some thoughts on this idea before actually implementing anything.

In a CI environment, it's useful to use --cache-from to select a set of images to consider for caching (that are pulled and not necessarily built on the box).

For usage with an internal registry, I might use something like this as a wrapper to docker build:

docker build $(docker images -q -a 'my-registry.com/*' | xargs -n1 echo --cache-from) ...

This effectively expands to:

docker build --cache-from ... --cache-from ... --cache-from ... --cache-from ... ...

While this is workable, it's a bit low on the quality-of-life scale.

I'm proposing an interface similar to this:

docker build --cache-from='my-registry.com/*' ...

This would consider all images tagged with my-registry.com/* as part of the trusted cache set (without having to manually enumerate them myself).

Thoughts?

thaJeztah commented 6 years ago

Looks like this was discussed in https://github.com/moby/moby/issues/33002#issuecomment-299041162, and https://github.com/moby/moby/issues/33002, and https://github.com/moby/moby/issues/32612#issuecomment-294052399

ping @tonistiigi

Spenser309 commented 6 years ago

Hey this would be a great feature. I have a environment where users use pull requests to get things into mainline. This means we create a new tag for each branch (and build number) like so.

PR-1 = user/imagename:PR-1

and the master branch will push to

master = user/imagename:latest

Right now we have --cache-from=user/imagename:latest on all nodes but because the tag from the branch is pushed to PR-1. The PR-1 image does not get used as a cache source for the master build. Right now to get this to work, I'd have to determine which PR was merged into the master branch or find a way to list all PR's as cache sources. It would be much simpler if I could just say --cache-from=user/imagename:* .

asottile commented 6 years ago

@Spenser309 you can probably replicate that today with:

 docker images --filter reference=user/imagename --format='{{.ID}}'
Spenser309 commented 6 years ago

@asottile ,

That seems to be just listing my local images. PR-1 may be built on a different machine than master. I'd like to be able to use the remote PR-1 image build as a cache.

asottile commented 6 years ago

Yeah, you'd have to docker pull the targets first