Support for new artifact types over OCI

jdolitsky commented 5 years ago

Hello, this is related to https://github.com/goharbor/harbor/issues/7773 (cc @reasonerjt)

Harbor should introduce support for new artifact types over OCI.

Some examples may include Helm charts, CNAB bundles, OPA bundles, as well as other artifact types that are sure to arise out of the cloud-native ecosystem. You can see a complete list of all artifact types that are recognized by Azure Container Registry (ACR) here.

As Harbor is built atop Docker Distribution, this is actually already supported out-of-the-box.

The only difference at a low-level is in the image manifest's config.mediaType field. Here is a snip of a Docker image manifest:

{
    "schemaVersion": 2,
    "config": {
        "mediaType": "application/vnd.docker.container.image.v1+json", // <-----
        "size": 7023,
        "digest": "sha256:b5b2b2c507a0944348e0303114d8d93aaaa081732b86451d9bce1f432a537bc7"
    },
    "layers": [
...

Here is a example of a Helm chart stored in distribution:

{
    "schemaVersion": 2,
    "config": {
        "mediaType": "application/vnd.cncf.helm.chart.config.v1+json", // <-----
        "size": 2,
        "digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a"
    },
    "layers": [
...

Harbor may wish to restrict allowed values for mediaType. This may be something that can even be configured on distribution (or should be). Defining which artifacts are supported will allow Harbor to build rich UIs atop specific supported artifacts as it has done with Helm charts.

jdolitsky commented 5 years ago

I've opened a PR on docker/distribution to support whitelisting artifact types: https://github.com/docker/distribution/pull/2934

Your "+1" there may be helpful to move this along :)

jdolitsky commented 5 years ago

After some experimentation, Harbor does indeed support this out of the box (at least from the registry point of view). We are able to push/pull Helm charts with manifest media type application/vnd.oci.image.manifest.v1+json (config media type application/vnd.cncf.helm.config.v1+json).

In order to make things show up in the database and UI, there is still some work to be done. This is the bare minimum small change to allow Helm 3 charts into database on helm chart push:

https://github.com/jdolitsky/harbor/commit/2f1953016cc4c2ccf141f301e8f97c4bc40d45a5#diff-351699f1d8dfe3bb64c4cb6b389e2ea6

eshepelyuk commented 5 years ago

Any feedback on this from Harbor team ?

xaleeks commented 5 years ago

@eshepelyuk We are already working with Josh and other MS guys involved with CNAB project on this. I'll update here when we have a more concrete release plan.

eshepelyuk commented 5 years ago

Thanks @xaleeks

jdolitsky commented 5 years ago

Please note that conversations for the proper ways to store artifacts over OCI has been moved to the new OCI Artifacts repo: https://github.com/opencontainers/artifacts

OrlinVasilev commented 1 year ago

@jdolitsky - not sure if that was resolved or not in the way you mentioned but is that part of the solution you were looking for? https://goharbor.io/docs/main/administration/user-defined-oci-artifact/ I think we can close this now ?

goharbor / community

Support for new artifact types over OCI #79