`cache-from: type=gha` does not find image in `COPY --from` directive

[lgrosz]

Behaviour

When building a dependent image in a second job, where the first job should cache a built image, the second build fails to find first job's image. I also attempted this like this, but it seems buildx cannot be used, in particular the build-push-action, with the same results.

The goal is to build a tools image in the first job and (eventually) several images which depend on the tools in the first image in the second job. The tools image should not be pushed to a public repository.

Steps to reproduce this issue

.                                    
├── tools                                                                                                       
│   └── Dockerfile                                                                                              
└── uses-tools                                                                                                  
    └── Dockerfile

• tools/Dockerfile

FROM ubuntu:22.04
RUN touch /tools

• uses-tools/Dockerfile

FROM ubuntu:22.04
COPY --from=tools:latest /tools /

Expected behaviour

Dockerfile under uses-tools context should successfully build.

Actual behaviour

Second job fails as it cannot find the (should be) previously cached image to copy from.

Configuration

• Repository URL
• Build URL

name: Build images
on:
  workflow_dispatch:

jobs:
  build-tools:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout the repo
        uses: actions/checkout@v3

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Build tools image
        uses: docker/build-push-action@v4
        with:
          tags: tools:latest
          context: tools
          push: false
          load: true
          cache-to: type=gha,mode=max

  build-uses-tools:
    needs: build-tools
    runs-on: ubuntu-latest

    steps:
      - name: Checkout the repo
        uses: actions/checkout@v3

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Build uses tools image
        uses: docker/build-push-action@v4
        with:
          context: uses-tools
          push: false
          load: true
          cache-from: type=gha

Logs

logs_41.zip

[crazy-max]

Using load: true will just load the image to the Docker store in build-tools job and will be lost when the job is done: https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions#the-components-of-github-actions

Each job will run inside its own virtual machine runner, or inside a container, and has one or more steps that either run a script that you define or run an action, which is a reusable extension that can simplify your workflow.

If you want to use an image in subsequent steps you need to use named contexts as shown in https://docs.docker.com/build/ci/github-actions/named-contexts/ and to share this image to another job, you can look at https://docs.docker.com/build/ci/github-actions/share-image-jobs/.

Btw I'm not sure why you need two distinct jobs to do this. A single job is enough for what you're doing imo so this example https://docs.docker.com/build/ci/github-actions/named-contexts/#use-image-in-subsequent-steps should be what you want.

[lgrosz]

@crazy-max

Btw I'm not sure why you need two distinct jobs to do this. A single job is enough for what you're doing imo so this example https://docs.docker.com/build/ci/github-actions/named-contexts/#use-image-in-subsequent-steps should be what you want.

The second job is going to be a matrix which each entry will use the same image produced in the first job. I didn't think it'd be great to build the same image n times over (but maybe it is, correct me if I'm wrong).

If you want to use an image in subsequent steps you need to use named contexts as shown in https://docs.docker.com/build/ci/github-actions/named-contexts/ and to share this image to another job, you can look at https://docs.docker.com/build/ci/github-actions/share-image-jobs/.

I was unaware of named contexts. I guess I just didn't search the right keywords. I'll give that a go.

Thank you for the timely response!

[lgrosz]

This may be getting out of scope of my original issue, but is it possible to upload the registry service container, as an artifact, so I can pull it down in subsequent jobs, like is able to be done with specific images? I'd like to do it by using the workflow services instead of running docker explicitly for those containers.

By the way this works great if the goal is to rebuild the "tools" image for every strategy in the matrix.

[lgrosz]

I ended up with the following...

name: Build images

on:
  workflow_dispatch:

jobs:
  build-tools:
    runs-on: ubuntu-latest
    services:
      registry:
        image: registry:2
        ports:
          - 5000:5000
        volumes:
          - /tmp/registry:/var/lib/registry

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
        with:
          # network=host driver-opt needed to push to local registry
          driver-opts: network=host

      - name: Build tools image
        uses: docker/build-push-action@v4
        with:
          context: tools
          tags: localhost:5000/tools:latest
          push: true

      - name: Upload registry as artifact
        uses: actions/upload-artifact@v3
        with:
          name: docker-registry
          path: /tmp/registry

  build-images:
    needs: build-tools
    runs-on: ubuntu-latest

    steps:
      - name: Download registry from artifacts
        uses: actions/download-artifact@v3
        with:
          name: docker-registry
          path: /tmp/registry

      # This must be done without the service container feature since the
      # volume needs to be populated before starting the container
      - name: Start conatiner registry
        run: |
          docker run --rm --detach --publish 5000:5000 \
            --volume /tmp/registry:/var/lib/registry \
            registry:2

      - name: Checkout
        uses: actions/checkout@v3

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v2

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
        with:
          # network=host driver-opt needed to push to local registry
          driver-opts: network=host

      - name: Build uses tools
        uses: docker/build-push-action@v4
        with:
          context: uses-tools
          build-contexts: |
            tools=docker-image://localhost:5000/tools:latest
          push: false

This works for my requirements. Feel free to drop suggestions. Note that service containers cannot be used in the registry in the second job as that would require populating the mount before the service container starts (which I don't think is possible at the moment). Closing this issue.