Helm template Release.namespace resolution fails on remote clusters

[beltran-rubo]

What steps did you take: Deploy a Carvel application with helmTemplate approach in a remote target cluster has a misconfigured namespace.

What happened: We created a Carvel package using the helmTemplate feature for Bitnami Cassandra. This Helm chart uses the namespace to create specific service names. The Release.namespace is a built-in object in Helm, in this specific case the Helm chart uses this variable.

Whenever we deploy the Carvel package in the same cluster that kapp-controller is running, the application works properly. The service uses the proper namespace for the service name and for the configuration of the application.

If we deploy the application in a remote cluster, different cluster that kapp-controller is running. The resolution of the Release.namespace is different and the service name does not match with the specified namespace into the PackageInstall CR. In this case the pods are looking for a service name that does not exist:

Seed provider couldn't lookup host cassandra-0.cassandra-headless.deployment-service.svc.cluster.local Meanwhile the pods configuration contain: /etc/hosts 192.168.63.24 cassandra-0.cassandra-headless.target-namespace.svc.cluster.local cassandra-0

The workaround for this is to specify the namespace into the Package CR into the helmTemplate section. But that is not ideal, users should be able to specify the target namespace into the PackageInstall CR.

What did you expect: Deploying a Carvel package in a remote cluster should use the same namespace to resolve the Helm chart variables.

Environment:

• kapp Controller version): 0.32.0
• ghcr.io/vmware-tanzu/carvel-kapp-controller@sha256:8e56eb65782697dd7aef3d9ab1df3294ade42ca41c5ecb38ff2cd579b3221d82
• Kubernetes version (use kubectl version): 1.22.5 (TKG v1.5.1)

[cppforlife]

The resolution of the Release.namespace is different and the service name does not match with the specified namespace into the PackageInstall CR.

what is the namespace that gets picked?

[beltran-rubo]

It uses the namespace where the App is created by default (deployment-service) into the 'centralised' cluster where kapp-controller is running (not the target cluster one).

[joe-kimmel-vmw]

@beltran-rubo Can you help us understand the impact of this bug on your workflow? It sounds like you have a workaround but would prefer a different workflow - is this critical or blocking progress for you?

[beltran-rubo]

Thanks @joe-kimmel-vmw right now we are using the workaround for the internal verification but thinking on the users of those Carvel packages would be complex to modify the Package CR just for specifying a namespace.

[benmoss]

Sounds like we can pass the PackageInstall namespace through to the Helm template at least in this case