Apply user-defined metadata (labels and annotations) when creating the release namespace

[dlipovetsky]

User Story

As a user, I would like to add metadata to the release namespace, as required by some charts [1]. A helm chart does not add metadata to the release namespace, for the same reason it does not create the release namespace.

[1] The speaker pod requires elevated permission in order to perform its network functionalities. If you are using MetalLB with a kubernetes version that enforces pod security admission (which is beta in k8s 1.23), the namespace MetalLB is deployed to must be labelled... -- https://metallb.org/installation/#installation-with-helm

Detailed Description

When CAAPH creates the release namespace, it should apply user-defined metadata (labels and annotations) to the namespace.

Anything else you would like to add:

Alternatives

1. Create the release namespace (with metadata) on the workload cluster, before creating the CAAPH resources for the release. If the cluster is based on a ClusterClass, this can be done using an AfterControlPlaneInitialized lifecycle hook.
2. Use a separate chart that creates a namespace, and then use that namespace as the release namespace.
3. Define a helm pre-install hook. This is a Kubernetes Job that helm creates before it creates the other resources in the release. The Job must create the namespace.

Implementation

CAAPH delegates namespace creation to the helm client. This client does not add metadata to the namespace, and helm maintainers have made it clear that this will not change. To be able to add metadata to the namespace, CAAPH would have to implement its own namespace creation.

/kind feature

[Jont828]

@dlipovetsky This is an interesting idea. You're correct in that currently, we just pass through the spec.namespace field to mimic the --create-namespace option in Helm install. How exactly do you propose to configure the namespace metadata? Is the idea to pass through all the labels/annotations on the HelmChartProxy to the namespace? Or to add a new API field in the spec?

[dlipovetsky]

Thanks for triaging!

Is the idea to pass through all the labels/annotations on the HelmChartProxy to the namespace?

I hadn't considered passing through labels/annotations on the HelmChartProxy. I would recommend against it, mainly because it would effectively become part of the API, but lack discoverability, and versioning, among other things.

Or to add a new API field in the spec?

I think an API field would make sense.

[dlipovetsky]

I'd like to hear from other users who need this; are they happy with one or more of the alternatives I listed above? Do they have other alternatives? I wouldn't want CAAPH to expand its API without a clear need.

[Jont828]

I hadn't considered passing through labels/annotations on the HelmChartProxy. I would recommend against it, mainly because it would effectively become part of the API, but lack discoverability, and versioning, among other things.

That makes sense, it would be a clever workaround but perhaps a bit too clever to be easily discovered by users.

I'd like to hear from other users who need this; are they happy with one or more of the alternatives I listed above? Do they have other alternatives? I wouldn't want CAAPH to expand its API without a clear need.

I haven't heard of anyone else who has this use case. Maybe worth bringing it up in Slack or CAPI office hours?

[shineeng]

I'd like to hear from other users who need this; are they happy with one or more of the alternatives I listed above?

we're currently going with 2, a separate chart that creates namespaces along with their metadata. I think this is somehow a bootstrapping issue (you can't lift yourself pulling your shoes) and I'm also confused by the somewhat contrasting informations I've found on my path from https://github.com/helm/helm/issues/3503 to here.

I'm no Helm expert, but I can't figure out why I can create a ns simply adding an option to helm install, while I can't remove one with helm uninstall --remove-namespace. There must be reasons for this I ignore, here comes my asymmetry feeling.