search

bpfman / bpfman

An eBPF Manager for Linux and Kubernetes
https://bpfman.io
Apache License 2.0
481 stars 42 forks source link

OCI-Image spec conformance #1162

Open Silvanoc opened 2 months ago

Silvanoc commented 2 months ago

Is there a reason why bpfman does not conform with the OCI-image specification?

I mean, it is still using the tool-specific Docker specification, instead of the standardized OCI-image specification.

Other tools using container registries as storage (Helm, cosign, notary,...) are relying on the OCI-image specification. Not only that, some of them are relying on OCI-image v1.1 features.

With OCI-image v1.1 features you can specify the type of the artifact (something like application/vnd.bpfman.package or similar) and of the different provided files (something like application/vnd.bpfman.bytecode).

Silvanoc commented 2 months ago

FYI the format that you are using was already being considered legacy by Docker 3 years ago.

Silvanoc commented 2 months ago

Command for diagnosing: regctl manifest get --format raw-body quay.io/bpfman-bytecode/go-tracepoint-counter|jq -r '.mediaType'

What returns application/vnd.docker.distribution.manifest.v2+json.

astoycos commented 2 months ago

Hiya @Silvanoc! Thanks for opening this, as it's the perfect time for your feedback/ help here while I'm updating the spec in https://github.com/bpfman/bpfman/pull/1141

Is there a reason why bpfman does not conform with the OCI-image specification?

The goal was to eventually ONLY use the OCI-image specification, however originally I ran into issues similar to what WASM images deal with.. i.e there was limitations with existing tooling + registries which is why I focused on writing the backwards compatible spec first. I've been trying to avoid having to implement the image build logic from scratch in bpfman at least at first. However I'm open to scoping out that work. The other issue is that we'd need registry support for custom artifactTypes, which I don't think quay supports at a generic level quite yet.

I mean, it is still using the tool-specific Docker specification, instead of the standardized OCI-image specification. Other tools using container registries as storage (Helm, cosign, notary,...) are relying on the OCI-image specification. Not only that, some of them are relying on OCI-image v1.1 features.

So when I was looking before at docker + podman there still wasn't any way to build generic images with custom mediaType and artifactType fields (as described in https://github.com/opencontainers/image-spec/blob/main/manifest.md#guidelines-for-artifact-usage) However I am just seeing the ORAS project tooling which might make this easier! https://oras.land/docs/installation

Above all I wanted to ensure that developers could easily package and deploy eBPF programs with existing tooling they're already familiar with.

Silvanoc commented 2 months ago

What WASM does with what they call the compat format is pretty much what you are doing as of now too, right? Smuggle your payload inside of a container image.

That format provides the best compatibility with old registries, but the worst feature set for new registries. Since registries cannot identify the artifact type in a standardize way, no advanced features (that I am sure are gonna get commodity) can be supported. Just think of a registry UI grouping the artifacts by type and providing a specific icon or similar features.

What they call the oci format is more or less what I would do to be future oriented. But a slight difference: use artifactType (manifest level) to specify the artifact type with something like application/vnd.bpfman.program.

There is an intermediate approach supported by

--> Use config/mediaType to specify the artifact type. Something that was not clearly specified on OCI-Image v1.0, but mostly supported since the Docker Registry supported it. It is now officially supported on the OCI-Image v1.1. But I consider it somehow a migration path. If I am right, thats what Helm is relying on.

BTW, I would recommend regctl over ORAS. It is very powerful and better aligned with the specification (the maintainer in one of OCI-spec maintainers). It can be used for managing low level elements (indexes, manifests, blobs) and high level elements (images, artifacts,...).

astoycos commented 2 months ago

What WASM does with what they call the compat format is pretty much what you are doing as of now too, right? Smuggle your payload into a container image.

100% correct, I got all the inspiration from them :)

That format provides the best compatibility with old registries, but the worst feature set for new registries. Since registries cannot identify the artifact type in a standardize way, no advanced features (that I am sure are gonna get commodity) can be supported. Just think of a registry UI grouping the artifacts by type and providing a specific icon or similar features.

I completely agree which is why I'd like to move towards the newer features, I'm just balancing that desire with time/priorities/other feature commitments.

What they call the oci format is more or less what I would do to be future oriented. But a slight difference: use artifactType (manifest level) to specify the artifact type with something like application/vnd.bpfman.program.

There is an intermediate approach supported by

  • all registries I am aware of
  • tools like regctl and ORAS

--> Use config/mediaType to specify the artifact type. Something that was not clearly specified on OCI-Image v1.0, but mostly supported since the Docker Registry supported it. It is now officially supported on the OCI-Image v1.1. But I consider it somehow a migration path. If I am right, thats what Helm is relying on.

Perfect I'm definitely alright with using config/mediaType for now.

BTW, I would recommend regctl over ORAS. It is very powerful and better aligned with the specification (the maintainer in one of OCI-spec maintainers). It can be used for managing low level elements (indexes, manifests, blobs) and high level elements (images, artifacts,...).

Sounds good I've never played with it before :) but now I'm excited to, I wish they had a rust SDK as that would make things super easy, currently we use https://github.com/krustlet/oci-distribution (which is moving to the ORAS project) for all interactions with container registries.

One of the other larger problems we're facing with eBPF images and the fact that we were using docker/podman for local image caching and therefore couldn't build/run bytecode images locally. i.e a user has to build, push, and then run since bpfman can't directly access the local cache in an abstracted way for both docker and podman.