Proposal: Adding extension API support

[sajayantony]

As a follow up of the the extension proposal discussed during the OCI call the issue below outlines a proposal for an extensions model to distribution API. The goal here is to provide a reference implementation that would demonstrate adding capabilities to support scenarios like reference artifacts for an image.

opencontainers/distribution-spec#111 - extension proposal opencontainers/distribution-spec#302 - adds some description for adding dist extensions

https://github.com/opencontainers/distribution-spec/pull/111#issuecomment-867251252

/cc @stevvooe's @shizhmsft @mikebrow @mnltejaswini

Details of the design proposal

• Upated proposal from @mnltejaswini - https://hackmd.io/7uwG9y6bR8-XpDwlReK9RQ?view
• Initial Proposal from @shizhMSFT https://hackmd.io/1zvOVNsvQ3a9D0knPOBrZw?view

PRs showing the idea

Current PR - https://github.com/distribution/distribution/pull/3522

Earlier Iterations - https://github.com/distribution/distribution/pull/3522 https://github.com/shizhMSFT/distribution/pull/1 https://github.com/shizhMSFT/distribution/pull/2 ——

Distribution Extensions

This document describes the changes to the distribution to support the extension API that is proposed to the distribution specification. It also outlines some of the guidelines for the extension authors when it is added to the distribution.

Extensions Discovery APIs

As per the extension specification, the distribution MUST support two new routes to enable the discovery of the extensions that are registered with the distribution.

Registry level extensions may be discovered with a standard GET as follows.

GET /v2/_ext/

Repository level extensions may be discovered with a standard GET as follows.

GET /v2/{name}/_ext/

To enable these routes, the distribution is updated to add these corresponding routes names, descriptors, URLs and handlers under the V2 API. The route handlers uses two properties registryExtensions and repositoryExtensions of the registry application to serve these requests. These properties are set during the registration process of the extensions.

TODO: Pagination support for the discovery APIs is still pending.

Authoring Extensions

This section describes various aspects of adding new extensions to the distribution. They include configuration, registration, interfaces and some guiding principles for integrating with the distribution codebase.

Guiding Principles

When distribution code has to be updated to support the addition of a new extension, the following guiding principles can be used by the extension authors

• All the extension specific implementations SHOULD reside in the new package path that is named by its namespace. This package SHOULD be placed under the package github.com/distribution/distribution/v3/registry/extension
• If any of the existing packages has to be updated to support extensions like utilities or storage models, they SHOULD reside in the file extension.go under that corresponding package.
• It is highly recommended to not update any existing distribution interfaces as part of implementing extensions. For example interfaces defined in registry.go, manifests.go, blobs.go, tags.go etc. However, extensions can create new interfaces that compose the existing interfaces as needed.
• The distribution storage model has two types of blob storage. One is the global blob store that is shared by all repositories. The other blob store is scoped to a repository and is called a linked blob store. The linked blob store handles the link blobs that point to the blobs in the global blob store. These link blobs are organized under three different entity types called layers, manifests and tags. This storage model is internal to the distribution and thereby not exported outside the storage package. In cases where extensions need a handle to this linked blob storage, it is advisable to write utilities that create general blob storage abstractions over this linked blob storage instead of exposing the linked storage directly. It is also advisable to expose this linked blob storage for read only operations only to limit the change surface.
• If the extensions have to persist new data as part of their implementation, it is recommended to store that data in a separate container(folder) either under the root or a repository. It is advisable to not update the existing storage model either global or linked blob storage used by the distribution.
• The distribution's storage driver is available to every extension if it needs to persist any new data in the underlying storage. Although, the extension has an handle to the driver, it is highly recommend to interact with registry entities like Manifest, Tags, Repositories, Blob etc using the distribution provided abstractions in registry.go, manifests.go, blobs.go, tags.go rather than using the storage driver.

Extension Configuration

A new section named extensions is added to the distribution configuration. This is parsed to a type map[string]interface{} and is read as a map with extension namespace as the key and the rest of the configuration under it as a generic interface{} All the configuration specified under the extension namespace is considered as opaque and will be passed to the extension initializer during registration. The configuration can be structured to match the extension route or the extension spec as shown below

extensions:
  ns:
    ext1:
      - component1
      - component2
    ext2:
      component1:
        foo: bar

extensions section is optional.

Extension Interfaces

A new interface called ExtensionNamespace is declared to represent an extension namespace. Currently this interface defines two methods as shown below

// ExtensionNamespace is the namespace that is used to define extensions to the distribution.
type ExtensionNamespace interface {
    // GetRepositoryRoutes returns a list of extension routes scoped at a repository level
    GetRepositoryRoutes() []ExtensionRoute
    // GetRegistryRoutes returns a list of extension routes scoped at a registry level
    GetRegistryRoutes() []ExtensionRoute
}

The extension namespace SHOULD return the set of registry and repository scoped routes that will be handled by it. This interface is open and can be updated to include any generic methods that apply to all extensions. The type ExtensionRoute includes the route components for an extension route defined by the spec as shown below

// Route describes an extension route.
type ExtensionRoute struct {
    // Namespace is the name of the extension namespace
    Namespace string
    // Extension is the name of the extension  under the namespace
    Extension string
    // Component is the name of the component under the extension
    Component string
    // Descriptor is the route descriptor that gives its path
    Descriptor v2.RouteDescriptor
    // Dispatcher if present signifies that the route is http route with a dispatcher
    Dispatcher RouteDispatchFunc
}

Only routes that are created with a dispatcher will be considered as HTTP routes and will be registered in the distribution router. If a route doesn't define the dispatcher, it is considered as an extension to a capability like storage rather than a HTTP server. Irrespective of the dispatcher, all the routes will be stored and will be included as part of the extension discovery API response.

Extension Registration

The registration workflow for extensions follows the same pattern that is used to register storage drivers, middlewares etc. in the distribution. An extension creator/initializer is defined that will create a namespace of type ExtensionNamespace.

// InitExtensionNamespace is the initialize function for creating the extension namespace
type InitExtensionNamespace func(ctx c.Context, storageDriver driver.StorageDriver, options configuration.ExtensionConfig) (ExtensionNamespace, error)

The registration of an extension namespace includes the registration of the corresponding namespace initializer at startup using the go package init() function semantics. Because of this semantics, the package path of the extension should be imported in the main.go of the distribution.

The name of the namespace MUST be unique across all the extension namespaces that are registered with the distribution.

The name that is used to register an extension namespace should match with the name of the ns that is defined in the extension configuration.

Sample Extensions

• Tag History and List Manifests
• Coming Soon: ORAS Artifacts and Referrers

References

• Distribution Extension - Initial Proposal with architecture diagrams
• Extensions API for Distribution
• Extension Proposal to Distribution

[vbatts]

ohman. 👀

[sajayantony]

@mnltejaswini brought up a conversation about simplifying this further during a conversation and I believe we can try to match this with OCI Distribution spec of

extensions: 
  <name>
      - <ext>
        - <component>