search

oras-project / oras

OCI registry client - managing content like artifacts, images, packages
https://oras.land
Apache License 2.0
1.41k stars 171 forks source link

Support multi-arch image/artifact push #1053

Open sajayantony opened 1 year ago

sajayantony commented 1 year ago

What is the version of your ORAS CLI

1.0.0

What would you like to be added?

As artifacts become increasingly accepted, it would be useful for ORAS to support multi-arch artifacts as well. There are already some standard conventions of tools like homebrew implementing this.

Multi-arch manifest below - https://explore.ggcr.dev/?image=ghcr.io%2Fhomebrew%2Fcore%2Foras:1.0.1

{
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:d0e42e6f6d564a9d1e178e72b42ff12b7e2e3c36cd32b44b150f7465c171db41",
      "size": 1914,
      "platform": {
        "architecture": "arm64",
        "os": "darwin",
        "os.version": "macOS 12.6"
      },
      "annotations": {
        "org.opencontainers.image.ref.name": "1.0.1.arm64_monterey",
        "sh.brew.bottle.digest": "8690763d0a9d0be6574ee4cfdb8edbab302f232c8056bae2d0083f170719afed",
        "sh.brew.bottle.size": "3436422",
        "sh.brew.tab": "{\"homebrew_version\":\"4.1.4-6-ga8022e6\",\"changed_files\":[],\"source_modified_time\":1691075089,\"compiler\":\"clang\",\"runtime_dependencies\":[],\"arch\":\"arm64\",\"built_on\":{\"os\":\"Macintosh\",\"os_version\":\"macOS 12.6\",\"cpu_family\":\"dunno\",\"xcode\":\"14.2\",\"clt\":\"14.2.0.0.1.1668646533\",\"preferred_perl\":\"5.30\"}}"
      }
    },

Each binary is uploaded with the following manifest 

 oras manifest get --pretty ghcr.io/homebrew/core/oras@sha256:d0e42e6f6d564a9d1e178e72b42ff12b7e2e3c36cd32b44b150f7465c171db41
{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.oci.image.config.v1+json",
    "digest": "sha256:0e7e738a321110bc395c82868cd93c6dbd0e4afa7b8931f08a6039ed59544249",
    "size": 222
  },
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
      "digest": "sha256:8690763d0a9d0be6574ee4cfdb8edbab302f232c8056bae2d0083f170719afed",
      "size": 3436422,
      "annotations": {
        "org.opencontainers.image.title": "oras--1.0.1.arm64_monterey.bottle.tar.gz"
      }
    }
  ],
  "annotations": {
    "com.github.package.type": "homebrew_bottle",
    "org.opencontainers.image.created": "2023-08-07",
    "org.opencontainers.image.description": "OCI Registry As Storage",
    "org.opencontainers.image.documentation": "https://formulae.brew.sh/formula/oras",
    "org.opencontainers.image.license": "Apache-2.0",
    "org.opencontainers.image.ref.name": "1.0.1.arm64_monterey",
    "org.opencontainers.image.revision": "7716dfd65ccfe0502f5acfc35b3e8c6aa4cc52b1",
    "org.opencontainers.image.source": "https://github.com/homebrew/homebrew-core/blob/7716dfd65ccfe0502f5acfc35b3e8c6aa4cc52b1/Formula/oras.rb",
    "org.opencontainers.image.title": "oras 1.0.1.arm64_monterey",
    "org.opencontainers.image.url": "https://github.com/oras-project/oras",
    "org.opencontainers.image.vendor": "homebrew",
    "org.opencontainers.image.version": "1.0.1",
    "sh.brew.bottle.digest": "8690763d0a9d0be6574ee4cfdb8edbab302f232c8056bae2d0083f170719afed",
    "sh.brew.bottle.size": "3436422",
    "sh.brew.tab": "{\"homebrew_version\":\"4.1.4-6-ga8022e6\",\"changed_files\":[],\"source_modified_time\":1691075089,\"compiler\":\"clang\",\"runtime_dependencies\":[],\"arch\":\"arm64\",\"built_on\":{\"os\":\"Macintosh\",\"os_version\":\"macOS 12.6\",\"cpu_family\":\"dunno\",\"xcode\":\"14.2\",\"clt\":\"14.2.0.0.1.1668646533\",\"preferred_perl\":\"5.30\"}}"
  }
}

Why is this needed for ORAS?

ORAS does not have CLI commands to author a manifest. Being able to author this multi arch manifest or index-based manifest would enable artifact authors to use ORAS CLI for scenarios like above.

Things to keep in mind

  1. Multi-arch platform property is one way for clients to determine the artifact. This is already supported in ORAS throught the --platform flag.
  2. Clients can choose to use annotations or any custom filter from index.
  3. There could be an ability for clients to compose the manifest using a tag and add more artifacts in a multi step script and finally copy the full index and related manifests in one shot.

Composotion

sequenceDiagram
    participant User
    participant LocalRegistry as Local Store
    participant RemoteRegistry as Remote Registry

    User->>LocalRegistry: Begin creation of artifact with tag mybinary:v1 for amd64 linux
    Note right of LocalRegistry: Artifact: mybinary:v1 (linux/amd64) initiated

    User->>LocalRegistry: Add binary for a different platform (e.g. arm64)
    Note right of LocalRegistry: Artifact: mybinary:v1 (linux/amd64, linux/arm64) updated

    User->>LocalRegistry: Add binary with a different annotation
    Note right of LocalRegistry: Artifact: mybinary:v1 (linux/amd64, linux/arm64, annotations)

    User->>RemoteRegistry: Copy artifact mybinary:v1 from Local Registry
    Note right of RemoteRegistry: Artifact: mybinary:v1 (linux/amd64, linux/arm64, annotations)

Appending

Consider appending a remote manifest and add a new item to the index 

sequenceDiagram
    participant User
    participant RemoteRegistry1 as Local Store
    participant RemoteRegistry2 as Remote Registry

    User->>RemoteRegistry1: Inspect artifact with tag mybinary:v1
    Note right of RemoteRegistry1: GET manifest details of mybinary:v1

    User->>User: Determine modifications without pulling full artifact

    User->>RemoteRegistry1: Add binary for a different platform (e.g. linux/arm64) directly
    Note right of RemoteRegistry1: Artifact: mybinary:v1 (linux/amd64, linux/arm64)  updated

    User->>RemoteRegistry: Copy artifact mybinary:v1 from Local Registry
    Note right of RemoteRegistry2: Artifact: mybinary:v1 (linux/amd64, linux/arm64) updated

Note

/cc @siggy

shizhMSFT commented 1 year ago

/cc @FeynmanZhou @yizha1

FeynmanZhou commented 1 year ago

There is another request for attaching artifacts to a multi-arch image with ORAS raised by an ORAS user @SamirPS described in https://github.com/oras-project/oras/issues/813. Designing a full user story for multi-arch image distribution (pull, push, discover, attach) in ORAS sounds reasonable to me.

qweeah commented 1 year ago

There is another request for attaching artifacts to a multi-arch image with ORAS raised by an ORAS user @SamirPS described in #813. Designing a full user story for multi-arch image distribution (pull, push, discover, attach) in ORAS sounds reasonable to me.

813 requires ability to find the correct subject, this issue is proposing ability of packing(composing or authoring) an index, which is different.

qweeah commented 1 year ago

Composing an index is easy and the composed index can be output to stdout for uploading like:

oras index my.reg/repo tag1 tag2 dgst2 --subject tag3 | oras manifest push -

Platform info can be easily obtained from the predecessor manifest's config, the tricky part is customizing descriptor annotations in manifests. e.g. we can found that the annotations of oras mac build doesn't all presented in the referring index.

I don't have any idea to accommodate the need for selectively add annotation into the index, I suggest we can introduce the index composing feature first.

qweeah commented 1 year ago
  1. For composition scenario, oras should be able to let user specify platform info when it's not in the artifact.
  2. oras push and oras attach should support --platform to allow user bake artifacts with platform metadata, see https://github.com/oras-project/oras/issues/1066
  3. For appending scenario, oras should not do garbage collection.
sajayantony commented 2 months ago

Adding 2 scripts to show the challenges https://gist.github.com/sajayantony/0c9d6436c03d531b1bbebe43249381cf and from @theory https://gist.github.com/theory/7dc164e5772cae652d838a1c508972ae#file-push_trunk

TerryHowe commented 2 months ago

Blog post https://justatheory.com/2024/06/trunk-oci-poc/

qweeah commented 1 month ago

This special case of composition scenario might need a index merge command:

The tool chain I used is listed as below

✗ docker version
Client: Docker Engine - Community
 Version:           27.1.0
 API version:       1.46
 Go version:        go1.21.12
 Git commit:        6312585
 Built:             Fri Jul 19 17:43:11 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.1.0
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.12
  Git commit:       a21b1a2
  Built:            Fri Jul 19 17:43:11 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.19
  GitCommit:        2bf793ef6dc9a18e00cb12efb64355c2c9d5eb41
 runc:
  Version:          1.1.1
  GitCommit:        v1.1.0-20-g52de29d7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

I run docker build and docker save and got below manifest:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:f4c8fbb580a9de23437857b686a96394451e5bd66c464ba025d09c8aee3de9d6",
      "size": 665,
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:cea0a71c4a8214c54e655c587607687a112ab1ed852a68add9863fbbb1b93774",
      "size": 566,
      "annotations": {
        "vnd.docker.reference.digest": "sha256:f4c8fbb580a9de23437857b686a96394451e5bd66c464ba025d09c8aee3de9d6",
        "vnd.docker.reference.type": "attestation-manifest"
      },
      "platform": {
        "architecture": "unknown",
        "os": "unknown"
      }
    }
  ]
}

Worth pointing out that even though I was trying to build a single arch image, still the builder(buildx) generates me an index, whose manifests field points to the image manifest and attestation manifest. The experience should apply to all users building from non-overlayfs image store (in my case, containerd image store)

To better help such user compose multi-arch image from those indexes. We can add a new command to accept index references and flatten the input into one index. Comparing to index create, the advantages are

  1. Performance is better: as it requires less network request to fetch platform information from the config
  2. Annotations added by building tools (like vnd.docker.reference.digest and vnd.docker.reference.type) will be kept.