`garden delete service` does not delete service from K8s' `default` namespace

[worldofgeese]

Bug

A module of type kubernetes will not delete on run of garden delete service.

Current Behavior

Expected behavior

Garden should delete the service when garden delete service is called, even if deployed to the default namespace.

Reproducible example

Use a minimal Garden project and manifests included at the following gist: https://gist.github.com/worldofgeese/7fb8aa3a771e10dd01774ffe466d41cf

Workaround

Manually delete resources with kubectl: kubectl delete -f cockroachdb.yml

Your environment

• OS: Windows 11 with WSL 2
• How I'm running Kubernetes: Civo remote K3s cluster

garden version 0.12.47

[vvagaytsev]

@worldofgeese is this still an issue in 0.13?

[worldofgeese]

@vvagaytsev Yep, just tested this with my reproducible example. It's a gist cloneable locally with git clone https://gist.github.com/7fb8aa3a771e10dd01774ffe466d41cf.git undeletable-service. A garden.yml and project.garden.yml are included, they're just not immediately visible without scrolling because of how gists sort files. See logs below the jump

```console ❯ garden deploy -l3 [verbose] garden version: 0.13.7 Deploy Garden v0.13 (Bonsai) is a major release with significant changes. Please help us improve it by reporting any issues/bugs here: https://go.garden.io/report-bonsai → Run garden util hide-warning 0.13-bonsai to disable this warning. ℹ garden → Running in Garden environment default.default ℹ providers → Getting status... ℹ providers [verbose] → resolve provider exec is ready. ℹ providers [verbose] → resolve provider container is ready. ℹ providers [verbose] → resolve provider templated is ready. ℹ providers [verbose] → resolve provider local-kubernetes is ready. ✔ providers → Cached (took 1.5 sec) ℹ providers → Run with --force-refresh to force a refresh of provider statuses. ℹ graph → Resolving actions and modules... ✔ graph → Done (took 0.1 sec) ℹ deploy.my-cockroachdb → deploying ℹ deploy.my-cockroachdb → Deploying version v-1c9a868c84... ℹ deploy.my-cockroachdb → Waiting for resources to be ready... ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Waiting for resources to be ready... ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of PodDisruptionBudget my-cockroachdb-budget is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of ServiceAccount my-cockroachdb-rotate-self-signer is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of ServiceAccount my-cockroachdb is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of ClusterRole my-cockroachdb is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of ClusterRoleBinding my-cockroachdb is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of Role my-cockroachdb-rotate-self-signer is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of Role my-cockroachdb is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of RoleBinding my-cockroachdb-rotate-self-signer is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of RoleBinding my-cockroachdb is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of Service my-cockroachdb is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of Service my-cockroachdb-public is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ⚠ deploy.my-cockroachdb → StatefulSet/my-cockroachdb: Pulling image "cockroachdb/cockroach:v22.2.0" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] StatefulSet/my-cockroachdb: Pulling image "cockroachdb/cockroach:v22.2.0" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of CronJob my-cockroachdb-rotate-self-signer is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of CronJob my-cockroachdb-rotate-self-signer-client is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of ServiceAccount my-cockroachdb-self-signer is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of Role my-cockroachdb-self-signer is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of RoleBinding my-cockroachdb-self-signer is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of Job my-cockroachdb-self-signer is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of Job my-cockroachdb-self-signer-cleaner is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of Job my-cockroachdb-init is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of ConfigMap garden-meta-deploy-my-cockroachdb is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ⚠ deploy.my-cockroachdb → StatefulSet/my-cockroachdb: Started container db ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] StatefulSet/my-cockroachdb: Started container db ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ⚠ deploy.my-cockroachdb → StatefulSet/my-cockroachdb: Readiness probe failed: HTTP probe failed with statuscode: 503 ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] StatefulSet/my-cockroachdb: Readiness probe failed: HTTP probe failed with statuscode: 503 ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ⚠ deploy.my-cockroachdb → StatefulSet/my-cockroachdb: Started container db ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] StatefulSet/my-cockroachdb: Started container db ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "deploying" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Status of StatefulSet my-cockroachdb is "ready" ℹ deploy.my-cockroachdb [verbose] → [kubernetes-plugin] Resources ready ℹ deploy.my-cockroachdb → Resources ready ✔ deploy.my-cockroachdb → Done (took 75.1 sec) ℹ deploy.my-cockroachdb [verbose] → Deploy type=kubernetes name=my-cockroachdb (from module my-cockroachdb) is ready. Done! [ble: elapsed 82.644s (CPU 12.2%)] garden deploy -l3 ☸ minikube (default) in undeletable-service on  main [!?] on ☁️ taohansen (eu-central-1) [4h27m40s] on ☁️ tao@garden.io took 3m54s ❯ garden delete service Cleaning up deployment(s) Garden v0.13 (Bonsai) is a major release with significant changes. Please help us improve it by reporting any issues/bugs here: https://go.garden.io/report-bonsai → Run garden util hide-warning 0.13-bonsai to disable this warning. ℹ garden → Running in Garden environment default.default ℹ providers → Getting status... ✔ providers → Cached (took 1.6 sec) ℹ providers → Run with --force-refresh to force a refresh of provider statuses. ℹ graph → Resolving actions and modules... ✔ graph → Done (took 0.1 sec) ℹ deploy.my-cockroachdb → Cleaning up... ✔ deploy.my-cockroachdb → Done (took 0.4 sec) ℹ deploy.my-cockroachdb → Done! Done! ☸ minikube (default) in undeletable-service on  main [!?] on ☁️ taohansen (eu-central-1) [4h26m25s] on ☁️ tao@garden.io took 1m15s ❯ kubectl get po -A NAMESPACE NAME READY STATUS RESTARTS AGE default my-cockroachdb-0 1/1 Running 0 6m9s default my-cockroachdb-1 1/1 Running 0 6m9s default my-cockroachdb-2 1/1 Running 0 6m9s default my-cockroachdb-init-hzprc 0/1 Completed 0 6m9s default my-cockroachdb-self-signer-cleaner-gqhnw 0/1 Completed 0 6m9s default my-cockroachdb-self-signer-rfkjm 0/1 Completed 0 6m9s fastapi-mongodb-default my-mongodb-c4f95879b-w66pn 1/1 Running 0 3h32m ingress-nginx ingress-nginx-admission-create-hgwbj 0/1 Completed 0 26h ingress-nginx ingress-nginx-admission-patch-qshfq 0/1 Completed 1 26h ingress-nginx ingress-nginx-controller-6cc5ccb977-mnhfl 1/1 Running 0 26h kube-system coredns-787d4945fb-l2rbr 1/1 Running 0 26h kube-system etcd-minikube 1/1 Running 0 26h kube-system kube-apiserver-minikube 1/1 Running 0 26h kube-system kube-controller-manager-minikube 1/1 Running 0 26h kube-system kube-proxy-x2xl2 1/1 Running 0 26h kube-system kube-scheduler-minikube 1/1 Running 0 26h kube-system storage-provisioner 1/1 Running 1 (26h ago) 26h templated-k8s-container-default my-service-5dc9784cd7-2mllc 1/1 Running 0 26h templated-k8s-container-default my-service-5dc9784cd7-bbk6s 1/1 Running 0 26h templated-k8s-container-default my-service-5dc9784cd7-mxhn9 1/1 Running 0 26h ```

[vvagaytsev]

Ok, thanks for confirming this @worldofgeese!