search

concourse / registry-image-resource

a resource for images in a Docker registry
Apache License 2.0
89 stars 107 forks source link

v1.9.0 broke pushing multi-architecture images #345

Open ChrisHines opened 1 year ago

ChrisHines commented 1 year ago

Describe the bug

The support for pushing multi-arch OCI images added in https://github.com/concourse/registry-image-resource/pull/321 seems to be broken in the new v1.9.0 release published yesterday.

Reproduction steps

  1. Build a multi-arch image with concourse/oci-build-task
  2. Push it with a put step using concourse/registry-image-resource:1.9.0 (specifying the directory created by the above step as described in https://github.com/concourse/registry-image-resource/pull/321.
  3. Observe the following error: 
    ERRO[0000] could not load image from path 'image/image': read /tmp/build/put/image/image: is a directory
  4. Pin to concourse/registry-image-resource:1.8.0
  5. Observe pushing the image now works.

Expected behavior

concourse/registry-image-resource:1.9.0 should work with multi-arch images just as the prior release, or there should be documentation to explain the necessary pipeline changes to get it to work.

Additional context

No response

neodem commented 11 months ago

I ran into this same issue. Based on the documentation and the demonstrated functionality of the resource that it expects to see the image param point to an actual tarball.. ex:

      - put: docker-hub
        params:
          image: image/image.tar

however, it seems that previous versions would accept an exploded tarball as well:

      - put: docker-hub
        params:
          image: image/image
        get_params:
          format: oci

(the above works in 1.8.0 but not in later versions).

The problem is that the oci-build-task does not include the manifest in its outputted tarball. I would say that's the original sin here.

tgoodsell-tempus commented 11 months ago

@neodem FYI, the exclusion of manifest.json is not a mistake, the current spec for image layouts suggests that the old manifest.json becomes embedded in blobs, while the OCI spec has specific index.json and oci-layout files at the root.

See: https://github.com/opencontainers/image-spec/blob/main/image-layout.md

Therefore, at least based on this information, registry-image should not fail on OCI built images where that manifest.json is not in the TAR/image root.

tgoodsell-tempus commented 11 months ago

Related upstream issue: https://github.com/google/go-containerregistry/issues/1756

My investigations so far lend me to an early conclusion that the tar reader for images is hardcoded to the "single arch image" style output.

tgoodsell-tempus commented 11 months ago

Also see: https://github.com/google/go-containerregistry/issues/1793

tgoodsell-tempus commented 11 months ago

FYI: https://github.com/google/go-containerregistry/issues/1809

tgoodsell-tempus commented 11 months ago

FYI, I have confirmed things are working for my sample use case on Concourse 7.9.1, hardcoding the registry image resource to 1.9.0:

See:

resource_types:
  - name: registry-image
    type: registry-image
    privileged: true
    check_every: 24h
    source:
      repository: concourse/registry-image-resource
      tag: 1.9.0

resources:
  - name: test-pipeline
    type: git
    check_every: 24h
    webhook_token: concourse
    source:
      uri: git@github.com:example/example.git
  - name: test-image
    type: registry-image
    source:
      repository: us-docker.pkg.dev/example/fly
      username: example
      password: example
jobs:
  - name: test-pipeline
    plan:
      - get: test-pipeline
        trigger: true
      - task: build-image-task
        privileged: true
        config:
          platform: linux
          image_resource:
            type: registry-image
            source:
              repository: concourse/oci-build-task
          inputs:
          - name: test-pipeline
          outputs:
          - name: image
          params:
            CONTEXT: test-pipeline/exampleDockerfile
            IMAGE_PLATFORM: linux/arm64,linux/amd64
            OUTPUT_OCI: true
          run:
            path: build
      - put: test-image
        params:
          image: image/image
          version: 0.0.1
ChrisHines commented 11 months ago

@tgoodsell-tempus Thanks for looking into this issue and sharing what you found. It is interesting that it works for you. Unfortunately I don't see many differences between your pipeline above and my pipeline that broke with 1.9.0. Here are the non-cosmetic differences I do see: