search

julia-actions / cache

A shortcut action to cache Julia artifacts, packages, and registries.
MIT License
38 stars 9 forks source link

Set `cache-registries` to true by default on non-Windows operating systems #7

Closed rikhuijzer closed 9 months ago

rikhuijzer commented 2 years ago

This was partially discussed in #2.

Currently, the cache-registries input is false by default because it may actually slow down running time on Windows. Many people will probably not notice this input and leave it on false or can't be bothered to ensure that this input is only set to true on non-Windows operating systems.

So, should we check whether the user didn't set anything and when the operating system is Ubuntu or macOS, then set cache-registries: true?

giordano commented 2 years ago

The problem with Windows is that the julia-buildpkg and julia-runtest workflows unconditionally set JULIA_PKG_SERVER="", so that the registry is always a bloated 200+ MB git repositories with 30k+ small files, which is horribly slow for Windows. By caching that, you put that in a tarball, but Windows is also horribly slow at unpacking tarballs with lots of small files, so that caching the registry in practice saves you nothing on that platform. The situation is much different on Unix systems, where tarballs are handled just fine.

If those workflows didn't set JULIA_PKG_SERVER="", caching the registry would make a bit more on all platforms, especially with Julia v1.7+ where the registry is just another tarball, although the size of the registry in this case would be of the order of 4 MB.

rikhuijzer commented 2 years ago

Thanks again for the clarification

If those workflows didn't set JULIA_PKG_SERVER="", caching the registry would make a bit more on all platforms, especially with Julia v1.7+ where the registry is just another tarball, although the size of the registry in this case would be of the order of 4 MB.

In a test workflow that I ran, restoring the artifacts, packages and registries directories took 6 seconds on an Ubuntu Julia 1.7 runner. Putting all this stuff into the cache took 8 seconds on an earlier Ubuntu run. So, that's pretty quick. For now, however, it seems the safest bet indeed to not cache the registries by default. I'll probably do some benchmarks at a later point to see what's better where.

giordano commented 2 years ago

https://github.com/JuliaPhysics/Measurements.jl/runs/4650042302?check_suite_focus=true#step:4:1 1m 22s on Windows, 15 s on macOS, 4s on Linux

IanButterworth commented 1 year ago

I think this can be enabled now.

https://github.com/julia-actions/julia-buildpkg/pull/31 - landed properly on https://github.com/julia-actions/julia-buildpkg/releases/tag/v1.4.2

https://github.com/julia-actions/julia-runtest/pull/74 - landed properly on https://github.com/julia-actions/julia-runtest/releases/tag/v1.9.1