Open ehuss opened 5 years ago
dependencies are often cached in Docker images
Is it actually a frequent occurrence? I know the playground does it and is a big public face of this, but are there other folk?
Docker zeros out the nanoseconds portion of the mtime when the image is saved.
Back in the long ago, we actually scraped through Cargo's files, resetting the nanoseconds; I don't suppose any such hackery would make sense still, or a similar form in a more supported manner? For example, CARGO_NANOSECOND_GRANULARITY=false
.
"Often" is an overstatement, but I do see people here and on forums who at least try to cache deps in docker images.
There have been a number of problems with mtimes in general. I'd like to see some kind of alternate change-tracking mode that makes it more reliable, but it's not clear exactly what that would look like.
The fingerprint of a build script is not included for dependencies. #6720 added this tracking, but because it requires mtimes to work correctly, #6734 removed it for registry dependencies. This is because dependencies are often cached in Docker images, and Docker zeros out the nanoseconds portion of the mtime when the image is saved.
This only affects the scenario when two separate commands are run, such as
cargo build
thencargo test
. The first command will pick up the changes, but the second will not.There are a few ways that I can think of that this would occur: changing an environment variable tracked by
rerun-if-env-changed
, a system library changes that is tracked byrerun-if-changed
, or adding a build override after already building.I cannot think of an easy way to fix this. I'd rather not try to change the mtime resolution in the fingerprint. I think the ideal solution would be: instead of tracking the
invoked.timestamp
, include the same fingerprint information calculated bybuild_script_local_fingerprints
when computing dependencies, and use a hybrid mtime/content hash forrerun-if-changed
files.Below are some tests that demonstrate this behavior. Both of these will fail on the last step if #6734 is merged.