`make deploy-local` fails if there is no `vsecm-system` namespace present.

[v0lkan]

Steps to Repo

Start with a fresh cluster.

Then, execute the following:

make k8s-delete
make k8s-start
eval $(minikube -p minikube docker-env)
make build-local
make deploy-local

Expectation:

• vsecm-system namespace should be created
• vsecm-system namespace should have Safe and Sentinel up and running w/o any issues.

What happens:

I get the following error log that indicates that the process cannot find vsecm-system namespace"

aegis@aegis:~/WORKSPACE/VSecM$ make deploy-local
kubectl apply -f ./k8s/0.21.5/crds
customresourcedefinition.apiextensions.k8s.io/clusterfederatedtrustdomains.spire.spiffe.io created
customresourcedefinition.apiextensions.k8s.io/clusterspiffeids.spire.spiffe.io created
customresourcedefinition.apiextensions.k8s.io/clusterstaticentries.spire.spiffe.io created
customresourcedefinition.apiextensions.k8s.io/controllermanagerconfigs.spire.spiffe.io created
kubectl apply -f ./k8s/0.21.5/0.21.5-local-distroless.yaml
namespace/spire-system created
serviceaccount/spire-agent created
serviceaccount/spire-server created
configmap/spire-agent created
configmap/spire-controller-manager-config created
configmap/spire-bundle created
configmap/spire-server created
clusterrole.rbac.authorization.k8s.io/secret-readwriter created
clusterrole.rbac.authorization.k8s.io/manager-role created
clusterrole.rbac.authorization.k8s.io/spire-agent-cluster-role created
clusterrole.rbac.authorization.k8s.io/spire-server-cluster-role created
clusterrolebinding.rbac.authorization.k8s.io/secret-readwriter-binding created
clusterrolebinding.rbac.authorization.k8s.io/manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/spire-agent-cluster-role-binding created
clusterrolebinding.rbac.authorization.k8s.io/spire-server-cluster-role-binding created
role.rbac.authorization.k8s.io/leader-election-role created
role.rbac.authorization.k8s.io/spire-server-role created
rolebinding.rbac.authorization.k8s.io/leader-election-rolebinding created
rolebinding.rbac.authorization.k8s.io/spire-server-role-binding created
service/spire-server created
service/spire-server-bundle-endpoint created
service/spire-controller-manager-webhook-service created
daemonset.apps/spire-agent created
deployment.apps/spire-server created
csidriver.storage.k8s.io/csi.spiffe.io created
clusterspiffeid.spire.spiffe.io/vsecm-safe created
clusterspiffeid.spire.spiffe.io/vsecm-sentinel created
validatingwebhookconfiguration.admissionregistration.k8s.io/spire-controller-manager-webhook created
namespace/vsecm-system created
Error from server (NotFound): error when creating "./k8s/0.21.5/0.21.5-local-distroless.yaml": namespaces "vsecm-system" not found
Error from server (NotFound): error when creating "./k8s/0.21.5/0.21.5-local-distroless.yaml": namespaces "vsecm-system" not found
Error from server (NotFound): error when creating "./k8s/0.21.5/0.21.5-local-distroless.yaml": namespaces "vsecm-system" not found
Error from server (NotFound): error when creating "./k8s/0.21.5/0.21.5-local-distroless.yaml": namespaces "vsecm-system" not found
Error from server (NotFound): error when creating "./k8s/0.21.5/0.21.5-local-distroless.yaml": namespaces "vsecm-system" not found
Error from server (NotFound): error when creating "./k8s/0.21.5/0.21.5-local-distroless.yaml": namespaces "vsecm-system" not found
make: *** [makefiles/VSecMDeploy.mk:41: deploy-local] Error 1

Also, although the namespace is there, there are no pods in it:

aegis@aegis:~/WORKSPACE/VSecM$ k get ns
NAME              STATUS   AGE
default           Active   26m
kube-node-lease   Active   26m
kube-public       Active   26m
kube-system       Active   26m
spire-system      Active   10m
vsecm-system      Active   10m
aegis@aegis:~/WORKSPACE/VSecM$ k get po -n vsecm-system
No resources found in vsecm-system namespace.
aegis@aegis:~/WORKSPACE/VSecM$ k get deployment -n vsecm-system
No resources found in vsecm-system namespace.

[rohatcan]

I can work on this

[rohatcan]

I might have found the issue when we add this annotations value, helm template adds the vsecm-system Namespace resource to the bottom of the manifest. When I removed the podAnnotations value from values.yml and generated the manifests, deployment worked as expected.

Previously, the podAnnotations key was duplicated, so I guess it was overriding, and it had no effect.

podAnnotations:
  "helm.sh/hook": pre-install

https://github.com/vmware-tanzu/secrets-manager/blob/main/helm-charts/0.21.5/values.yaml#L60-L61

[v0lkan]

@rohatcan it’s all yours. have fun :) .

[v0lkan]

Working as expected right now.

Closing.

Please let me know if it needs to be reopened.