docker / buildx

Docker CLI plugin for extended build capabilities with BuildKit
Apache License 2.0
3.56k stars 481 forks source link

Question: Using a local and remote cache for docker-image-layers in Github Actions #1443

Open JakeSummers opened 1 year ago

JakeSummers commented 1 year ago

I am trying to get Docker-image-layer caching working for my github environment.

One quirk of my build system is that I (indirectly) call docker build twice.

Unfortunately, when I setup gha caching, this doesn't significantly speed up my build because for the second call to docker build it pulls from the gha cache instead of the local cache.

It looks like you can only push to a single cache so I cannot use a local & remote cache. Are there any workarounds to this issue?

Github action file:

name: Build 

on:
  push:
    branches: [ "**" ]

jobs:
  build_deploy
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      # Needed for docker layer caching
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      # Needed for docker layer caching
      - name: Expose GitHub Runtime
        uses: crazy-max/ghaction-github-runtime@v2

      - name: Quality Checks
        run: make run-checks-docker     # <- This does a docker build

      - name: Deploy
        run: make deploy                # <- This does a docker build

Makefile:

run-checks: docker-build
    docker run $(DOCKER_IMAGE_NAME) pytest

deploy: docker-build
    docker run \
        $(DOCKER_IMAGE_NAME) \
        poetry run python -m my.code.here

docker-build:
    docker buildx create --use --driver=docker-container ; \
    docker buildx build \
    --cache-to type=gha \
    --cache-from type=gha \
    --load \
    --tag $(DOCKER_IMAGE_NAME) \
    --file build.Dockerfile . ; \
jonathantorley commented 1 year ago

I think in this case the call to docker buildx create --use is throwing away the current buildx driver (which has been configured by crazy-max/ghaction-github-runtime@v2) and is replacing it with a fresh, unconfigured one. But that may be incorrect.