search

earthly / lib

Mozilla Public License 2.0
7 stars 11 forks source link

global caches & cargo-sweep #20

Closed idelvall closed 10 months ago

idelvall commented 10 months ago

Summary

This PR:

On global cache

In order to make the new global cache work with sharing=shared mode (that allows parallel builds to access its contents), I had to make sure that $CARGO_HOME/.package-cache was also being stored in the cache, since this the file used by Cargo as a lock to synchronize the access to $CARGO_HOME from parallel builds.

The simplest way I found to achieve this was by temporarily changing the whole $CARGO_HOME location to a folder within the cache, while running the same bin/cargo program.

idelvall commented 10 months ago

Ok, by using global caches (caches with id) I encounter this exact problem https://users.rust-lang.org/t/does-dockers-mount-type-cache-cause-problems/97073. Now trying to understand why cargo locks don't work in the buildkit cache mount...

idelvall commented 10 months ago

I've checked and flock properly works on shared cache mounts, so still investigating why I am facing this https://github.com/rust-lang/cargo/issues/11376

idelvall commented 10 months ago

This looks promising: https://github.com/rust-lang/cargo/issues/11376#issuecomment-1314542696

idelvall commented 10 months ago

This last commit sets $CARGO_INSTALL_ROOT pointing to the original $CARGO_HOME (in the target filesystem, not the mount cache).

The reason is that, installed crates are not fetched on demand if not found (they need to be installed), so if BK GC removes them from the cache then the build will fail, while if they are stored in the target layers and BK GCed them, the build will rerun those commands and install them back.

This setting makes cargo install to store the crates in the target filesystem to avoid such scenario