search

reproducible-containers / buildkit-cache-dance

Save `RUN --mount=type=cache` caches on GitHub Actions ( Forked from https://github.com/overmindtech/buildkit-cache-dance )
Apache License 2.0
98 stars 25 forks source link

Large cache takes a long time to inject/extract #13

Open alex-statsig opened 8 months ago

alex-statsig commented 8 months ago

Mostly a discussion point, but I've noticed large caches take a very long time to inject/extract. In my case, I have a 467MB cache of npm modules. This is downloaded from github's cache into the workflow runner in 8s, and thus is often worth caching for GH workflows. However, the injection step takes 53s. It seems like a lot of this time is transferring context, supposedly 1.8GB worth (taking 30s). I'm not sure if this is because the context is no longer zipped like when GH downloads it, or if something else is causing the 500MB to triple in size. The extraction step takes 2m56s. Happy to provide any logs as needed.

ermakovich commented 4 months ago

@alex-statsig caching NPM modules usually negatively affects build performance, because you spend more time on extracting and saving cache back compared to just not using cache and simply downloading packages from npm registry each time. NPM modules consist from many files, that's why cache extraction/saving is expensive.