search

joshuanianji / idris-2-docker

Multi-arch, multi-distro Docker images for Idris 2
MIT License
13 stars 2 forks source link

Improve cache #71

Closed joshuanianji closed 10 months ago

joshuanianji commented 10 months ago

Caching got improved a bit by #45, but it is still not as good as I would like it to be. We're encountering lots of cache misses and (i'm assuming) cache thrashing in the builtin github actions cache (since it's only 10GB). It shows up the most when I make a new PR since new branches don't seem to be hitting cache from main branch.

There are a couple of ideas I have right now:

  1. cache-from the built image in the ghcr registry as well (https://docs.docker.com/build/cache/backends/registry/)
  2. Don't cache-to when deploying. I think the arm layers are being set to the GHA cache, which is completely unnecessary (this is from looking at the docker build logs). The initial build-base and build-devcontainer are sufficient (and no need to cache ubuntu/debian images). This, the GHA cache should only be for amd builds UPDATE: cache-to with multi-stage builds are nondeterministic and results in a race condition, so best to avoid it (see: https://github.com/docker/buildx/discussions/1382#discussioncomment-6252049)
  3. Figure out how caching is done on the ARM64 side. I assumed it'll have it's own build-in cache that would work automatically since it's a buildx SSH endpoint to a cloud VM, but this doesn't seem to be working. On deployments, the ARM builds take the bulk of the time (which is annoying since it only has 3 cores). This might be solved by idea 1, but i'll have to try it out. UPDATE: Looking at the results for 5e802cb, it seems like idea 1 does work. The arm images are being cached.
  4. Run a remote registry I can use as a registry cache, instead of doing the above (might be fun but probably not worth it)

This PR also fixes a bug where the devcontainer would deploy to the idris-2-docker/base image lol. That would have been bad...