[proposal] Add CRDs from external helm chart

[eshepelyuk]

Hello

Wdyt about extending this controller with possibility to extract CRDs from published Helm charts repositories. (both OCI and plain old HTTP).

Since Helm has old issue that CRDs are not automatically upgraded but the CRDs are still packaged within Helm charts, this application could address this gap.

[Skarlso]

Oh that is a very good idea! Thanks for this proposal. I actually know how to do that. :D

💯

[eshepelyuk]

Oh that is a very good idea! Thanks for this proposal. I actually know how to do that. :D

💯

In terms of cli it would be smth like

helm pull --untar .... /tmpdir
.. do magic with crd folder ...

[Skarlso]

Yep, pretty much. Also, flux has some libraries in it's publish pkg repository to deal with OCI repos and extract tar files. So integrating that will help massively in providing this feature.

[eshepelyuk]

Yep, pretty much. Also, flux has some libraries in it's publish pkg repository to deal with OCI repos and extract tar files. So integrating that will help massively in providing this feature.

i just assumed helm libs itself should support oci repos, without even going to any 3rd party.

[Skarlso]

That's true. I thought you might want to do something like a helm source which would require credentials and would use helm package or something to extract / fetch the helm resources.

[Skarlso]

The difficulty here will be tag discovery. Since that is broken in Helm for OCI repositories. I'll see what I can do about it. Like, I can use oras or containerregistry or whatever to see which is the latest tag. And then hope it's a chart file. :D

[eshepelyuk]

The difficulty here will be tag discovery. Since that is broken in Helm for OCI repositories. I'll see what I can do about it. Like, I can use oras or containerregistry or whatever to see which is the latest tag. And then hope it's a chart file. :D

First, what do you mean broken and 2nd - why do you mention latest tag ?

Could you plz provide more details ?

[Skarlso]

https://github.com/helm/helm/issues/11000

I'm not saying latest; I'm saying latest as in the newest version. So v1.0.0 v2.0.0 v3.0.0. To detect if a new version has been pushed.

[eshepelyuk]

https://github.com/helm/helm/issues/11000

I'm not saying latest; I'm saying latest as in the newest version. So v1.0.0 v2.0.0 v3.0.0. To detect if a new version has been pushed.

sorry, i don't understand :(

[eshepelyuk]

helm should already support semantic ranges for OCI, so i still not understanding the issue.

[Skarlso]

Can it list tags for a specific chart?

I need to be able to track which is the latest version to reconcile new versions. If a new versions is available, I'm installing it.

In order to do that, I'm asking for a list of versions and see if the latest version is greater than the current reconciled versions given a semver constraint. So like, only patch versions, or, major, or minor.

[Skarlso]

For example:

helm search repo chart oci://ghcr.io/skarlso/helm/crd-bootstrap --versions

Returns

No results found

So I can't use helm to discover tags.

But that's fine.. I can use oras for example, or anything else that deals with OCI registries.

[eshepelyuk]

Can it list tags for a specific chart?

I need to be able to track which is the latest version to reconcile new versions. If a new versions is available, I'm installing it.

In order to do that, I'm asking for a list of versions and see if the latest version is greater than the current reconciled versions given a semver constraint. So like, only patch versions, or, major, or minor.

this is a very bad idea imo. the versions of CRDs must be driven by customer restriction, i.e. either particular version, or semantic range ( assuming helm golang library supports it, because helm CLI supports it)

[eshepelyuk]

For example:

helm search repo chart oci://ghcr.io/skarlso/helm/crd-bootstrap --versions

Returns

No results found

So I can't use helm to discover tags.

But that's fine.. I can use oras for example, or anything else that deals with OCI registries.

this is incorrect syntax, this should work

helm search repo crd-bootstrap -l

[Skarlso]

Did you try it? :) It does not work. :)

this is a very bad idea IMO. the versions of CRDs must be driven by customer restriction, i.e. either particular version, or semantic range ( assuming >helm golang library supports it, because helm CLI supports it)

Yes, this is exactly what I just wrote. You define a semver. If the semver allows it it will update. If it doesn't it won't. :)

[eshepelyuk]

Did you try it? :) It does not work. :)

this is a very bad idea IMO. the versions of CRDs must be driven by customer restriction, i.e. either particular version, or semantic range ( assuming >helm golang library supports it, because helm CLI supports it)

Yes, this is exactly what I just wrote. You define a semver. If the semver allows it it will update. If it doesn't it won't. :)

i am AFK, command lines writing by memory. but about latest - there should not be latest, semver exact or range should be set. and helm already handles this.

[Skarlso]

I'm not looking at latest. I'm looking for new versions that are allowed by a set semver.

[eshepelyuk]

I'm not looking at latest. I'm looking for new versions that are allowed by a set semver.

since i am afk - cant suggest smth in particular

but let give you an example.

I was using helm library chart as a dependency for my charts.

That helm library chart was published to github packages, i.e OCi registry.

I was declaring the dependency with semantic version, like ~6.

So, every time i was running helm dep up withing my my charts, the latest semantic version was resolved and downloaded.

[Skarlso]

Aha. Interesting. I'll take a look at that code section how they circumvent the search problem. Thanks for the pointer.

[Skarlso]

So dep update basically downloads everything again into the local registry and THEN selects the latest version using the semver. I could use something similar internally, but I don't require this level of engagement. I literally just need some tags from the OCI registry and that's it. :)

For plain HTTP and use helm search.

edit: technically it only downloads the index file. not the actual whole thing.

[eshepelyuk]

So dep update basically downloads everything again into the local registry and THEN selects the latest version using the semver. I could use something similar internally, but I don't require this level of engagement. I literally just need some tags from the OCI registry and that's it. :)

For plain HTTP and use helm search.

edit: technically it only downloads the index file. not the actual whole thing.

for plain HTTP it downloads index.yaml from the repo. for OCI it queries tags via OCI registry API.

[Skarlso]

for OCI it queries tags via OCI registry API.

Yep:

tags, err := c.RegistryClient.Tags(strings.TrimPrefix(ref, fmt.Sprintf("%s://", registry.OCIScheme)))

I guess I could use the registry client I already construct. Hopefully that works.

[Skarlso]

The latest release should contain this feature. v0.5.0.

[eshepelyuk]

The latest release should contain this feature. v0.5.0.

great. need a time to test it, hope during the weekend.

[eshepelyuk]

Maybe worth mentioning here ? https://github.com/helm/helm/issues/8668

[Skarlso]

Thanks. I'll take a look at it. 😊

[Skarlso]

Heh, interesting answer on that issue....

[eshepelyuk]

Heh, interesting answer on that issue....

which one ?

[Skarlso]

There is only one.

[Skarlso]

This is now done. Please feel free to re-open or create a new issue if problems arise. :) 🙇