Cache precompilation cache

giordano commented 2 years ago

It may also be useful caching .julia/compiled, to avoid re-precompiling packages, whose sources are already cached by default.

rikhuijzer commented 2 years ago

Yes I was also thinking about this. Could save minutes on jobs with many dependencies.

Do you know whether this folder can be re-used between different CPUs?

giordano commented 2 years ago

Uhm, there may be a problem for multi-architecture builds for same operating system: the slug depends, among other things, on the full absolute path of the sysimage and the julia executable, which would be different for different operating systems, but for example for x86 and x64 builds of Linux those paths might be the same (not 100% sure, you'd need to check)

giordano commented 2 years ago

In the worst case this can be disabled by default and the README would explain clearly that the precompilation cache can be used only if you don't have multi-architecture jobs for the same operating system.

DilumAluthge commented 2 years ago

Can we modify Julia to include the architecture when naming the precompilation files?

giordano commented 2 years ago

That would mean modifying Base.compilecache_path. It may be a good idea for the future, but won't apply to past versions even if accepted.

The easiest thing would be controlling where julia-actions/setup-julia installs Julia. I have no idea if that includes the architecture somewhere.

giordano commented 2 years ago

The easiest thing would be controlling where julia-actions/setup-julia installs Julia. I have no idea if that includes the architecture somewhere.

Looks like it does: https://github.com/julia-actions/setup-julia/blob/2d6959702913c06ee8c6997dadafe089ec18549f/src/setup-julia.ts#L62?

SaschaMann commented 2 years ago

Two things I'd be concerned about, without knowing details about what's stored in .julia/compiled and about how wrong files in that directory would affect things:

How will partial cache hits affect this, e.g. when some packages were cached but others weren't? Or one cache includes an older version? Does Pkg figure things out on its own in that case, too?
How will this work when several Julia versions are installed and used alongside each other in the same GHA environment? This could also be considered out of scope for this action fwiw.

The easiest thing would be controlling where julia-actions/setup-julia installs Julia. I have no idea if that includes the architecture somewhere.

setup-julia sets a julia-bindir output, e.g. /opt/hostedtoolcache/julia/nightly/x86/bin which includes that info. The workflow that calls both actions also contains the info about the architecture, so we could require it as an input for this action too. setup-julia already returns the installed Julia version as an output, so while it would be redundant we could also add a julia-arch output to it if that was useful.

(I don't know if you only need the information about the architecture or if the actual installation path influences how the cache would work. The suggestions above only work for the former.)

As a side note, we might also need to pay attention to the cache size and consider which directories yields the largest benefit when cached. The total storage across all caches in a repo is limited to 10 GB. This could be hit fairly quickly if the repo is active enough (=> there are a lot of branches and therefore caches) and/or the cached directories are large. To be clear, this is not necessarily an argument against adding this directory but it should be considered (by someone who knows more about Pkg internals/compilation etc than I do)

rikhuijzer commented 2 years ago

From a related PR by Dilum: Apparently caching the precompilation works except for Windows: https://github.com/JuliaLang/julia/pull/45552.

giordano commented 1 year ago

Can we have this, disabled by default, with a big warning about all the problems above? I have a few workflows that I'd like to speed up which I run on a single platform (typically Linux), so I don't care about possible conflicts.

DilumAluthge commented 1 year ago

Note that there are some other things you need to do to prevent the precompilation cache from being invalidated: https://github.com/julia-actions/setup-julia/pull/100#issuecomment-1198675969

giordano commented 1 year ago

Much awaited https://github.com/JuliaLang/julia/pull/47184 will also come with sensibly longer precompilation times, it'd be nice to reconsider this since it could save quite a lot of time 🙂

tfiers commented 1 year ago

Given that non-Windows, non-multi-architecture workflows probably cover the majority of Julia CI runs (see e.g, PkgTemplates), might this rather be enabled by default?

Or an "auto" option that detects whether it's safe to cache or not

Will indeed save lots of time and server-energy across the ecosystem

julia-actions / cache

Cache precompilation cache #11