search

dotnet / dotnet-buildtools-prereqs-docker

Used to maintain the Docker images hosted at the mcr.microsoft.com/dotnet-buildtools/prereqs image repository
MIT License
53 stars 99 forks source link

Cache misses from differing commit hash leading to unnecessary builds #1018

Open mthalman opened 2 months ago

mthalman commented 2 months ago

There are a few Dockerfiles which are not being recognized as a cache hit to an already published version of the image.

From the build log (internal link):

Image info's Dockerfile commit: https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/507c1c3291de8f64d9592f72ae92f8f9f0b45353/src/ubuntu/22.04/coredeps/Dockerfile
Latest Dockerfile commit: https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/70a3638e31ff3f7f69a9373b0b4c06773f5eaee6/src/ubuntu/22.04/coredeps/Dockerfile
Dockerfile commits match: False
CACHE MISS
Image info's Dockerfile commit: https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/3fc5553956a6e28906d81758fc2852c6a1fd93cc/src/ubuntu/18.04/crossdeps/Dockerfile
Latest Dockerfile commit: https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/70a3638e31ff3f7f69a9373b0b4c06773f5eaee6/src/ubuntu/18.04/crossdeps/Dockerfile
Dockerfile commits match: False
CACHE MISS
Image info's Dockerfile commit: https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/507c1c3291de8f64d9592f72ae92f8f9f0b45353/src/ubuntu/22.04/coredeps/Dockerfile
Latest Dockerfile commit: https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/70a3638e31ff3f7f69a9373b0b4c06773f5eaee6/src/ubuntu/22.04/coredeps/Dockerfile
Dockerfile commits match: False
CACHE MISS

These are being detected as being different commit hashes than what Dockerfile was last used to publish. But that's false because the different commit hashes are actually pointing to the same version of the Dockerfile. For example, with that first one (https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/70a3638e31ff3f7f69a9373b0b4c06773f5eaee6/src/ubuntu/22.04/coredeps/Dockerfile), if you go to the URL in your browser, you'll see the webpage refer to that version using the short hash of the equivalent hash:

image

It's not clear why there are differences in the values, nor why the publishing process doesn't update the image info JSON content with the value that we're detecting the file to have.

lbussell commented 2 months ago

[Triage] My suspicion is that something is up with this command: https://github.com/dotnet/docker-tools/blob/8fa03e26afbf7676865f2335c0b2f78baf5dbfe9/src/Microsoft.DotNet.ImageBuilder/src/GitHelper.cs#L43

Something must be causing it to retrieve the wrong commit. In the commit it returned, 70a3638e31ff3f7f69a9373b0b4c06773f5eaee6, there were no changes to the coredeps Dockerfile.