search

artifacthub / hub

Find, install and publish Cloud Native packages
https://artifacthub.io
Apache License 2.0
1.68k stars 229 forks source link

Add support for gadgets from Inspektor Gadget #3665

Closed alban closed 7 months ago

alban commented 8 months ago

Hello, I am one of the maintainers of the Inspektor Gadget project.

Is your feature request related to a problem? Please describe.

In the Inspektor Gadget project, users can publish their gadgets (containing ebpf programs) in OCI registries. I experimented with artifact-hub and I could publish a gadget (example: runc-vuln-gadget).

The problem is that it is displayed as a "container image". This is confusing because this is not something that can be executed with "docker run". We just use OCI registries to store our artifacts.

Describe the solution you'd like

Gadgets from Inspektor Gadget are displayed as "kind = gadget" instead of "kind = container image".

If Inspektor Gadget publishes its OCI images with annotation io.artifacthub.package.kind=gadget, could artifact-hub automatically figure out that it should not be displayed as "kind = container image"?

Describe alternatives you've considered

I don't know..

Additional context

https://github.com/inspektor-gadget/inspektor-gadget/issues/2175

tegioz commented 8 months ago

Hi @alban 👋

Inspektor gadgets would be a nice addition to Artifact Hub 🙂

Let me explain a bit how Artifact Hub indexes content.

Any organization or user can add repositories of any kind to Artifact Hub. At the moment we support 20+ repositories kinds, including Helm charts, OLM operators or the container image you've been experimenting with. The tracker component polls them periodically and collects metadata as needed.

Depending on the repository kind, metadata is extracted one way or another. For kinds that have already defined how a repository should be structured and served, like Helm charts or OLM operators, we have specialized tracker sources. This allowed publishers to start adding existing repositories without requiring any additional work. For other kinds, we have a generic tracker source based on our own metadata file. To keep AH as simple to maintain and extend as possible, we are trying to push this tracker as the recommended way to go whenever possible (most artifacts kinds supported use it actually -including Kubewarden policies, which are also stored in OCI registries-). In this particular case, it could be a great fit.

The generic tracker relies on a custom metadata file and a flexible directory structure that supports one or more packages per repository, including multiple versions per package if needed. Data unique to the artifact kind can be added in the form on custom annotations.

Some examples of how other projects organize the Artifact Hub metadata for their artifacts (they can be quite helpful for inspiration😇):

Please let us know if you have any questions! 🙂

alban commented 8 months ago

Thanks! Yes, we could use the artifacthub-pkg.yml metadata file.

Looking at the example from keptn, I am not sure about the annotation we would use to tell Artifact Hub that it is a gadget from Inspektor Gadget. Would the following be suitable? But Artifact Hub would need to be patched to tell it to look at that specific annotation.

annotations:
  inspektor-gadget/kind: "gadget"

It seems a bit weird to me that there is no general field kind.

tegioz commented 8 months ago

No worries!

You don't need to use an annotation to tell AH about the kind, that happens at the repository level. We'll add support for that new repository kind so that any user/org can add Inspektor Gadgets repositories (see the screenshot below). AH needs to be patched for every new kind supported: we need to register the new kind, extend some endpoints, add custom logos, documentation, update the home page, display custom information in the UI, etc 😅 But it's usually something minimal, it doesn't take long 😉

However, you can use annotations to include IG specific information (i.e. like the version the gadget is compatible with). Please see these ones from Headlamp plugins we've added recently. We can also display this information in the UI in a special way (only visible on IG packages).

Screenshot 2024-02-26 at 14 03 55
tegioz commented 8 months ago

I forgot: the Keptn kind you mentioned is an internal kind within Keptn, not an Artifact Hub kind. You could do something similar if you had multiple Gadgets kinds and wanted them displayed somehow in AH.

alban commented 7 months ago

Thanks.

So we would ask gadgets authors to add artifacthub-pkg.yml in their repositories. We can reuse the containersImages field, so Artifact Hub could give the correct links.

I think we can start without gadget specific annotations in artifacthub-pkg.yml. This could be added later once we get the basics right.

Some of the fields from our gadget.yaml metadata file are the same as in artifacthub-pkg.yml (name, description, etc.). To provide consistency, IG could have a tool to generate the artifacthub-pkg.yml from the gadget metadata.

We have the following logos: https://inspektor-gadget.io/media/brand-logo.svg https://inspektor-gadget.io/media/brand-icon.svg Do you need more?

tegioz commented 7 months ago

Cool, sounds great @alban 👍 We'll get it done soon (probably early next week, but maybe earlier if we find some time to work on this).

Quick question: by default, AH scans containers images for security vulnerabilities periodically using Trivy, and the results are published on the UI. But this doesn't make sense in some cases, depending on the content of the images. Would you like to have this feature disabled for gadgets?

alban commented 7 months ago

Yes we should disable Trivy for now.

Later, we could investigate it. I filed https://github.com/inspektor-gadget/inspektor-gadget/issues/2549 for this.

tegioz commented 7 months ago

Hi @alban

Quick question about metadata validation and signatures.

For Kubewarden policies, we enforce the following rules:

We would like to do something similar for Inspektor Gadgets, so it'd be good to know your thoughts about them in the context of IG. Also, any preference about the images names (i.e. gadget?)?

Thanks!

alban commented 7 months ago

Do you mean the following in artifacthub-pkg.yml?

containersImages:
  - name: gadget # mandatory
    image: ghcr.io/alban/runc-vuln-gadget:latest
    platforms:
      - linux/amd64
      - linux/arm64
  - name: gadget-alternative-location # optional
    image: alban.azurecr.io/runc-vuln-gadget:latest
    platforms:
      - linux/amd64
      - linux/arm64

I think your suggestions are good. And the names gadget and gadget-alternative-location are good too.

tegioz commented 7 months ago

Yes, awesome! 👍

Support for Inspektor Gadgets should be ready tomorrow 🙂

tegioz commented 7 months ago

This is ready @alban @mqasimsarfraz 🙂

You can see it in action in the staging environment, we'll deploy it to production soon.

https://staging.artifacthub.io/packages/search?kind=22 https://staging.artifacthub.io/docs/topics/repositories/inspektor-gadgets/

We've created a fork of the repository you shared to test it. Feel free to experiment with your own repositories if you'd like.

NOTE: it's possible to use a separate README.md file instead of embedding its content in the metadata file -it needs to be located in the same directory of the package metadata file though-.

Closed by #3693

alban commented 7 months ago

Thank you! I'll start experimenting with it :)

tegioz commented 7 months ago

Support for Inspektor Gadgets deployed to production!