docker / build-push-action

GitHub Action to build and push Docker images with Buildx
https://github.com/marketplace/actions/build-and-push-docker-images
Apache License 2.0
4.24k stars 541 forks source link

Built image is loaded but not visible to docker , so not available in the next step #892

Closed nvuillam closed 1 year ago

nvuillam commented 1 year ago

Behaviour

Steps to reproduce this issue

  1. Build MegaLinter image
  2. Run test cases on newly built image

Expected behaviour

Image should be available, but it is not.

Actual behaviour

The build actions triggers no error, as shown is the following log

#115 exporting to docker image format
#115 exporting layers
#115 exporting layers 230.5s done
#115 exporting manifest sha256:89a076476ec517f62b9543091381793a45ca276b2591a6f9a4bb4aa928b59677 done
#115 exporting config sha256:0d6a98afb11e313c04605abf51cafb1cbff4c2dc701ae4c450b69b71953a364d done
#115 sending tarball
#115 ...

#116 importing to docker
#116 DONE 134.7s

#115 exporting to docker image format
#115 sending tarball 187.3s done
#115 DONE 417.8s

image

But at the next step that needs to reuse the newly built image, it is not found.

docker image ls shows that the image is not available.

REPOSITORY                   TAG                                IMAGE ID       CREATED          SIZE
2a6c8c                       adf0f7b553354a8aafc8909dbb9e5da2   3c19552a810e   25 minutes ago   279MB
node                         16                                 d233337d62a4   11 days ago      909MB
node                         16-alpine                          d[22](https://github.com/oxsecurity/megalinter/actions/runs/5438434548/jobs/9889665936?pr=2767#step:14:23)e42cbfdbb   11 days ago      118MB
node                         18                                 a6cdafc3fbec   11 days ago      1.09GB
node                         18-alpine                          f85482183a4f   11 days ago      175MB
alpine                       3.15                               029bed813f07   2 weeks ago      5.59MB
alpine                       3.16                               5cb2da5c[23](https://github.com/oxsecurity/megalinter/actions/runs/5438434548/jobs/9889665936?pr=2767#step:14:24)91   2 weeks ago      5.54MB
alpine                       3.17                               1f73a9d63274   2 weeks ago      7.05MB
buildpack-deps               buster                             ed0e87ab36c0   2 weeks ago      802MB
buildpack-deps               bullseye                           36bc1816225d   2 weeks ago      833MB
debian                       10                                 49ae9acb6591   2 weeks ago      114MB
debian                       11                                 7df9c1a77ca2   2 weeks ago      1[24](https://github.com/oxsecurity/megalinter/actions/runs/5438434548/jobs/9889665936?pr=2767#step:14:25)MB
ghcr.io/aquasecurity/trivy   0.42.1                             95d6d6912185   3 weeks ago      212MB
ubuntu                       20.04                              6[26](https://github.com/oxsecurity/megalinter/actions/runs/5438434548/jobs/9889665936?pr=2767#step:14:27)a42b93d93   3 weeks ago      72.8MB
ubuntu                       22.04                              99284ca6cea0   3 weeks ago      77.8MB
ubuntu                       18.04                              f9a80a55f492   4 weeks ago      63.2MB
moby/buildkit                buildx-stable-1                    152ba3e57b49   2 months ago     168MB
moby/buildkit                latest                             152ba3e57b49   2 months ago     168MB
node                         14                                 1d12470fa662   2 months ago     912MB
node                         14-alpine                          0dac3dc[27](https://github.com/oxsecurity/megalinter/actions/runs/5438434548/jobs/9889665936?pr=2767#step:14:28)b1a   3 months ago     119MB
alpine                       3.14                               9e179bacf43c   3 months ago     5.61MB
tonistiigi/binfmt            latest                             [35](https://github.com/oxsecurity/megalinter/actions/runs/5438434548/jobs/9889665936?pr=2767#step:14:36)4472a[37](https://github.com/oxsecurity/megalinter/actions/runs/5438434548/jobs/9889665936?pr=2767#step:14:38)893   11 months ago    60.2MB

So logically, the following docker run fails

Unable to find image 'oxsecurity/megalinter:test-nvuillam-devskim-mypy' locally
docker: Error response from daemon: manifest for oxsecurity/megalinter:test-nvuillam-devskim-mypy not found: manifest unknown: manifest unknown.

Please not that the same pattern works very well on smaller images , like in this job https://github.com/oxsecurity/megalinter/actions/runs/5438434894/jobs/9889667424?pr=2767 , and always worked well on big images job

Configuration

      - name: Build MegaLinter Docker Image (full from main repo) & push
        if: |
          (
          (github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository) ||
          (github.event_name == 'push' && github.repository == 'oxsecurity/megalinter')
          )
          &&
          !contains(github.event.head_commit.message, 'quick build')
        uses: docker/build-push-action@v4
        with:
          context: .
          file: Dockerfile
          platforms: linux/amd64
          build-args: |
            BUILD_DATE=${{ env.BUILD_DATE }}
            BUILD_REVISION=${{ github.sha }}
            BUILD_VERSION=${{ steps.image_tag.outputs.tag }}
          load: true
          push: false
          secrets: |
            GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}
          tags: |
            oxsecurity/megalinter:${{ steps.image_tag.outputs.tag }}

      # Free disk space
      - name: Free Disk space
        shell: bash
        run: |
          sudo rm -rf /usr/local/lib/android  # will release about 10 GB if you don't need Android
          sudo rm -rf /usr/share/dotnet # will release about 20GB if you don't need .NET

      #####################################
      # Run Linter test cases #
      #####################################

      - name: Run Test Cases
        shell: bash
        run: |
          GITHUB_REPOSITORY=$([ "${{ github.event_name }}" == "pull_request" ] && echo "${{ github.event.pull_request.head.repo.full_name }}" || echo "${{ github.repository }}")
          GITHUB_BRANCH=$([ "${{ github.event_name }}" == "pull_request" ] && echo "${{ github.head_ref }}" || echo "${{ github.ref_name }}")

          export CI_ENV="$(bash <(curl -s https://codecov.io/env)) -e GITHUB_ACTIONS"
          TEST_KEYWORDS_TO_USE=""
          if [[ "${{ github.event.head_commit.message }}" == *"TEST_KEYWORDS="* ]]; then
            COMMIT_MSG="${{ github.event.head_commit.message }}"
            TEST_KEYWORDS_TO_USE=${COMMIT_MSG#*TEST_KEYWORDS=}
            echo "Run only tests with keywords ${TEST_KEYWORDS_TO_USE}"
            if [[ "${TEST_KEYWORDS_TO_USE}" =~ $'\r' ]]; then
                echo "Problem while parsing test keywords: switch back to all tests"
                TEST_KEYWORDS_TO_USE=""
            fi
          fi
          docker image ls
          docker run $CI_ENV -e TEST_CASE_RUN=true -e OUTPUT_FORMAT=text -e OUTPUT_FOLDER=${{ github.sha }} -e OUTPUT_DETAIL=detailed -e GITHUB_SHA=${{ github.sha }} -e GITHUB_REPOSITORY=${GITHUB_REPOSITORY} -e GITHUB_BRANCH=${GITHUB_BRANCH} -e GITHUB_TOKEN="${{ secrets.GITHUB_TOKEN }}" -e TEST_KEYWORDS="${TEST_KEYWORDS_TO_USE}" -e MEGALINTER_VOLUME_ROOT="${GITHUB_WORKSPACE}" -v "/var/run/docker.sock:/var/run/docker.sock:rw" -v ${GITHUB_WORKSPACE}:/tmp/lint oxsecurity/megalinter:${{ steps.image_tag.outputs.tag }}
        timeout-minutes: 90

Logs

logs_113764.zip

crazy-max commented 1 year ago

Looks like a duplicate of https://github.com/docker/build-push-action/issues/321

nvuillam commented 1 year ago

Thanks @crazy-max , i'll have a look:)

Maybe if there is an official workaround, it could be added in troubleshooting page ?