search

dotnet / docker-tools

This is a repo to house some common tools for our various docker repos.
MIT License
122 stars 46 forks source link

Build job optimization based on build cache #633

Open mthalman opened 4 years ago

mthalman commented 4 years ago

This is part of a set of phases outlined by https://github.com/dotnet/docker-tools/issues/186.

This phase would optimize the build for cases where a particular build job would end up not producing any new images because all the images it would have been result in cache hits. To do this, the same logic that the build command uses to determine whether an image needs to be built would be executed as part of the build matrix generation. When that logic is run over the entire set of Dockerfiles, any build legs that do not have any Dockerfiles to be built can simply be trimmed from the matrix. This avoids executing build jobs that would end up not building any new images. This is only intended as an optimization for trimming out such jobs. The build command would still be responsible for calculating which images need to be built on its own.

mthalman commented 3 years ago

In order to implement this, we need to ensure that generating the build matrix is not done in parallel with mirroring the base images. This could lead to incorrect caching results where an image is mistakenly thought to be up-to-date when in fact the base image tag had not been mirrored yet. See https://github.com/dotnet/dotnet-docker/pull/2617.