aiidalab / aiidalab-docker-stack

Docker images with the basic software stack for AiiDAlab
https://aiidalab.net
Other
9 stars 14 forks source link

Explore using cache for docker build #452

Open danielhollas opened 4 months ago

danielhollas commented 4 months ago
            set: |
                *.platform=linux/amd64
                *.output=type=registry,push-by-digest=true,name-canonical=true
                *.cache-from=type=local,src=/tmp/.buildx-cache
                *.cache-to=type=local,mode=max,dest=/tmp/.buildx-cache-new
            files: |
                docker-bake.hcl
                build.json

      # Temp fix for issue of fast size grow in local caching:
      # https://github.com/docker/build-push-action/issues/252
      # https://github.com/moby/buildkit/issues/1896
      - name: Move cache
        run: |
          rm -rf /tmp/.buildx-cache
          mv /tmp/.buildx-cache-new /tmp/.buildx-cache

Could you give it a try on caching? This may further speed the build?

_Originally posted by @unkcpz in https://github.com/aiidalab/aiidalab-docker-stack/pull/439#discussion_r1593629285_

danielhollas commented 4 months ago

Looks like a simpler solution is now available that uses GHA cache directly

https://github.com/restatedev/restate/pull/492/files

See also documentation here: https://docs.docker.com/build/ci/github-actions/cache/

danielhollas commented 4 months ago

Might not be so easy in the end: https://github.com/docker/bake-action/issues/87

unkcpz commented 4 months ago

Yep. I did a try in https://github.com/pspgen/build-machine/blob/478dbcebdc521d916f2c72f0500f67898d2a9f13/.github/workflows/ci.yml#L55-L73 ;) which borrow most of the idea from your PR. The build of my image takes 10 mins, using cache safe me half day at least 😄

danielhollas commented 4 months ago

So just to followup here, the cache works only partially. I suspect there is some cache-trashing since we're building multiple images and multiple platforms. So the cache wins us a bit, but could be much better. Perhaps worth looking into more in the future.

This issue is probably relevant https://github.com/docker/bake-action/issues/87 (note the last comment by Simon)