search

paritytech / cachepot

cachepot is `sccache` with extra sec, which in turn is `ccache` with cloud storage
https://cachepot.cc
Apache License 2.0
171 stars 13 forks source link

Workaround `cargo` fingerprinting `mtime` #165

Open TriplEight opened 2 years ago

TriplEight commented 2 years ago

Every time cache gets downloaded/copied/unzipped it receives a newer mtime than your git diff will ever be. It's not the case for the local caching obviously. This is majorly the case for all the CI use-cases both for sccache and cachepot, and for a long time. Moreover, both are showing the cache hits, but afterwards, all the cache is being invalidated.

The workaround we could come up with, would be to update mtime of the modified files so they are newer than the cache artifacts:

  1. git-restore-mtime
  2. find out the git diff
  3. touch the files were changed

Showcased in comparison with rusty-cachier in https://github.com/paritytech/cargo-contract/pull/635. cc @gilescope @rcny https://github.com/rust-lang/cargo/pull/8623 https://github.com/awslabs/smithy-rs/pull/1422

gilescope commented 2 years ago

To fix things properly on cargo / rustc my PRs would have to be broken down and landed in a series of bite sized increments. Doing it all as one big change was just too much. I have to say the build.rs shenanigans is a tad tricky and I don't think that's quite nailed yet in the cargo PR. The rustc changes were fine though and they could all be landed independently. The cool thing with the PR though is that almost all the required hashes are already available but just not made use of.

On Thu, 21 Jul 2022 at 17:13, Denis Pisarev @.***> wrote:

Every time cache gets downloaded/copied/unzipped it receives a newer mtime than your git diff will ever be. It's not the case for the local caching obviously. This is majorly the case for all the CI use-cases both for sccache and cachepot, and for a long time. Moreover, both are showing the cache hits, but afterwards, all the cache is being invalidated.

The workaround we could come up with, would be to update mtime of the modified files so they are newer than the cache artifacts:

  1. git-restore-mtime https://github.com/MestreLion/git-tools/blob/main/git-restore-mtime
  2. find out the git diff
  3. touch the files were changed

Showcased in comparison with rusty-cachier in paritytech/cargo-contract#635 https://github.com/paritytech/cargo-contract/pull/635. cc @gilescope https://github.com/gilescope @rcny https://github.com/rcny rust-lang/cargo#8623 https://github.com/rust-lang/cargo/pull/8623 awslabs/smithy-rs#1422 https://github.com/awslabs/smithy-rs/pull/1422

— Reply to this email directly, view it on GitHub https://github.com/paritytech/cachepot/issues/165, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGEJCAUP6PFHSIRPCKS3ILVVFZIZANCNFSM54IGYNCQ . You are receiving this because you were mentioned.Message ID: @.***>