gperdomor / nx-tools

Nx Workspaces builders and tools
MIT License
358 stars 56 forks source link

How to build an image of type application/vnd.docker.distribution.manifest.v2+json from GitHub pipeline? #1150

Open WTPOptAxe opened 1 month ago

WTPOptAxe commented 1 month ago

When I do a local build and push using INPUT_TAGS="1" yarn nx affected --target container, it creates an image of type application/vnd.docker.distribution.manifest.v2+json that I can push and then pull from my GCP repo.

If I do a build in GitHub, the resulting image is of type application/vnd.oci.image.index.v1+json

How can I change this to end up with the same kind of image in both cases?

Edit It looks to be related to https://github.com/docker/buildx/issues/1533 , but because I use nx for the container build, I can't work out how to pass -o oci-mediatypes=false

My github workflow is as follows:

- name: Configure Docker for Google Artifact Registry
        run: |
          gcloud auth configure-docker ourregion.pkg.dev

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - uses: actions/setup-node@v4
        with:
          node-version: '20.x'
          cache: 'yarn'

      - uses: actions/cache@v4
        with:
          path: '**/node_modules'
          key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}

      - name: Install dependencies
        run: yarn install --immutable

      - name: 'Build and push images'
        run: INPUT_GITHUB_TOKEN=${{ secrets.GH_ACCESS_TOKEN }}" yarn nx affected --target container --push
WTPOptAxe commented 1 month ago

The solution for me was to manually build using docker buildx build and disable OCI images. This is the relevant section of a working Github workflow

    - name: Determine Affected Projects
        id: get_affected_projects
        run: |
          affected_projects=$(yarn nx show projects --affected --base=${{ needs.set-nx-base.outputs.NX_BASE }} | tr '\n' ',' | tr -d '[:space:]')
          echo "AFFECTED_PROJECTS=${affected_projects}" >> $GITHUB_ENV

      - name: Build and push images
        run: |
          REPO="myregion.pkg.dev/myproject/myrepo"
          if [ -z "${{ env.AFFECTED_PROJECTS }}" ]; then
            echo "No affected projects to build."
          else
            for project in $(echo $AFFECTED_PROJECTS | tr ',' '\n'); do
              if [ -d "apps/${project}" ]; then
                echo ${project}
                DOCKER_BUILDKIT=1 docker buildx build --build-arg BUILDX_NO_DEFAULT_OCI=true --push --provenance=false -f apps/${project}/Dockerfile -t ${REPO}/${project}:${{ env.image_tag }} . --progress=plain --no-cache
              else
                echo "${project} not in apps - not building docker image"
              fi
            done
          fi
pgagnidze commented 3 weeks ago

@WTPOptAxe I have the same issue, I provided the provenance in the project.json:

"options": {
        "file": "path/to/Dockerfile",
        "provenance": "false",
        "platforms": ["linux/amd64"],
        "metadata": {
          "images": ["ghcr.io/<hidden>"]
        }
      },

I also tried from the GitHub Actions:

- name: Build Affected
        shell: bash
        env:
          INPUT_CACHE_FROM: 'type=gha,mode=max,scope=develop'
          INPUT_CACHE_TO: 'type=gha,mode=max,scope=develop'
          INPUT_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          INPUT_PUSH: true
          INPUT_TAGS: type=sha
          INPUT_PROVENANCE: false
        run: |
          yarn nx affected \
            --target=build \
          ...

Without any luck. I will try your workaround, which should work fine, but there should be better ways.