jonjohnsonjr
commented
5 years ago A lot of this seems more about listing than searching. We really need to figure out what that will look like. I had a strawman here that suffered a painful death by bikeshedding.
as results may be paged, sorting the results by name and/or version with ascending and defending options
descending
What's a "version" here, from a registry perspective? A tag or digest? Just a tag? A new thing?
Example: The helm cli would need to query a registry for charts that match a specific name. The result should return helm only artifacts.
This might be a pain. The artifacts stuff works because registries can be (mostly) agnostic to the contents of what's being distributed. The more stuff we require to be indexed, the less flexible we make the distribution API.
Search queries may specify date ranges, enabling the return of artifacts that have been created or changed since a given date:time
Artifact creation from the client's perspective (i.e. created time) or when it was pushed?
Search results shall be limited to the artifacts the user has read access control. The user may be a person or service account.
The spec doesn't currently speak to access models currently, so I'm not sure how appropriate this is.
SteveLasker
rchincha
vbatts
mikebrow
OCI Artifact Search Requirements
As registries support multiple artifact types, a search/catalog API that supports filtering on the artifact type will be needed.
The docker v1 registry spec supported Docker Search. While some vendors like Quay.io implemented the v1 search API, the majority of vendors require the v2 registry api which dropped search.
We believe revisiting the search api will support client CLIs that span registries, such as
helm search,duffle search(CNAB),docker search, and other evolving artifact types.By supporting a common search API across all registries, users could consistently use these new artifact CLIs across all registries.
This issue focuses on capturing the requirements for new Search and Eventing APIs. As the requirements are agreed upon, we'll move to a spec that captures the requirements.
Search is a generic capability used across several different use cases.
Tool Specific Searches
Helm, Singularity, Docker, OPA, CNAB and other tools will need to query their specific artifact types across various registries. Example: The helm cli would need to query a registry for charts that match a specific name. The result should return helm only artifacts.
Version specific searches:
Registry Specific Search
Users want to query registries for the artifacts that match a specific name or list artifacts within a given path. In this case, the results contain multiple artifact types.
Today, registries have created unique client APIs and server APIs. Until we have a generic registry client, it's expected registries will have vendor specific APIs. However, having common registry server side APIs expands the possibility for common tooling across registries.
A registry search API would include
Existing examples
ACR list repo example:
Without a common search/catalog API, cloud vendors have had to implement vendor specific experiences:
ACR list tags example, w/ future type added:
A repo could contain multiple artifact types
Rather than each registry vendor having to offer unique APIs, the goal would be to offer a common API.
Registry Tool Search - Scanners
Vendors and the community have attempted to build tools atop registries.
Without a common search/catalog API, these tools must work with individual images.
One of the most common registry tools include image scanning tools like Aqua, Twistlock, Neuvector and Clair. While the scanning tools protect runtime nodes, they all pre-scan registries to understand image vulnerabilities before they're run.
Scanners evaluate images in registries with a combination of a search/catalog API and events.
These vulnerability scanners need the following:
Today, scanners assume all artifacts in a registry are a container image. As a registry stores new artifact types, scanners will either need to know how to scan these new artifacts, or at least filter the results to artifacts they support.
Artifact Types
A registry must know the types it hosts for it to provide meaningful search results. Artifact types will be internally identified by an expanded set of OCI Media Types.
However, displaying
application/vnd.cncf.helm.chart.v1+jsondoes not make for a good user experience. To provide clean user experiences, a list of artifact types, a short description, and info on the artifact tooling will be maintained. Media Type Short Namesapplication/vnd.oci.image.index.v1+jsonapplication/vnd.oci.image.manifest.v1+jsonapplication/vnd.cncf.helm.chart.v1+jsonapplication/vnd.oci.cnab.index.v1+json* most registry providers automatically convert
oci.imagemanifests to the format requested by the client.Registry Search Requirements
Listing repos
Listing artifacts
Listing versions
Filtering by artifacts
Filtering by date ranges
Search queries may specify date ranges, enabling the return of artifacts that have been created or changed since a given date:time
manifest.configobjects. This allows a registry to option to add value, while not burdening all registries to parse all config objects of all artifact types.Paging
Results may be paged, to provide a full list of artifacts. A default page size of 100, with the ability to change the paging size.
Sorting
As results may be paged, being able to sort provides the ability to get the top n results, based on a given sort order. Sorting includes ascending and descending.
Role Based Access Control
Search results shall be limited to the artifacts the user has read access control. The user may be a person or service account. The spec shall not define specific rights or roles for how authorization should be implemented or managed, rather simply state the registry must be cognizant of security and support the security of it's product and/or platform. If a user has read access to repo1 and repo3, but not repo2, the repository listing should only return repo1, repo3. The spec will not define a role where read differentiates between management and data operations.