search

loft-sh / vcluster

vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
https://www.vcluster.com
Apache License 2.0
6.92k stars 426 forks source link

feat(cli): implement platform destroy command #2261

Closed rohantmp closed 16 hours ago

rohantmp commented 2 weeks ago

What issue type does this pull request address? (keep at least one, remove the others) /kind feature

What does this pull request do? Which issues does it resolve? (use resolves #<issue_number> if possible) resolves ENG-3980

Please provide a short message that should be published in the vcluster release notes Add a new cli command: vcluster platform destroy

What else do we need to know?

The command will wait for each resource type to finish cleaning up before proceeding to the next, so that resource types that depend on each other will not prevent cleanup. (Remove a cluster object before a virtualclusterinstance running on it, etc)

vcluster platform destroy (no args) on v4.0.0:

16:28:54 info found platform installation in namespace "vcluster-platform"
16:28:54 debug checking all "virtualclusterinstances"
16:28:54 info deleting virtualclusterinstances p-default/v1
16:28:56 debug checking all "virtualclusterinstances"
16:28:58 debug checking all "virtualclusterinstances"
16:29:00 debug checking all "virtualclusterinstances"
16:29:02 debug checking all "virtualclusterinstances"
16:29:04 debug checking all "virtualclusterinstances"
16:29:06 debug checking all "virtualclusterinstances"
16:29:08 debug checking all "virtualclusterinstances"
16:29:10 debug checking all "virtualclusterinstances"
16:29:12 debug checking all "virtualclusterinstances"
16:29:14 debug checking all "virtualclusterinstances"
16:29:14 debug legacy resource "virtualclusters" not found in discovery, skipping
16:29:14 debug checking all "devpodworkspaceinstances"
16:29:14 debug checking all "spaceinstances"
16:29:14 debug checking all "virtualclustertemplates"
16:29:14 info deleting virtualclustertemplates vcluster-pro-isolated-template
16:29:14 info deleting virtualclustertemplates vcluster-pro-template
16:29:16 debug checking all "virtualclustertemplates"
16:29:16 debug checking all "devpodenvironmenttemplates"
16:29:16 debug checking all "devpodworkspacetemplates"
16:29:16 info deleting devpodworkspacetemplates aws
16:29:16 info deleting devpodworkspacetemplates azure
16:29:16 info deleting devpodworkspacetemplates digitalocean
16:29:17 info deleting devpodworkspacetemplates ecs
16:29:17 info deleting devpodworkspacetemplates gcloud
16:29:17 info deleting devpodworkspacetemplates kubernetes
16:29:18 debug checking all "devpodworkspacetemplates"
16:29:18 debug checking all "clusterroletemplates"
16:29:18 info deleting clusterroletemplates loft-apps-admin
16:29:18 info deleting clusterroletemplates loft-apps-creator
16:29:19 info deleting clusterroletemplates loft-apps-viewer
16:29:19 info deleting clusterroletemplates loft-cluster-admin
16:29:19 info deleting clusterroletemplates loft-cluster-authenticated
16:29:19 info deleting clusterroletemplates loft-cluster-extra-space-admin-rules
16:29:19 info deleting clusterroletemplates loft-cluster-extra-space-viewer-rules
16:29:19 info deleting clusterroletemplates loft-cluster-space-admin
16:29:19 info deleting clusterroletemplates loft-cluster-space-viewer
16:29:19 info deleting clusterroletemplates loft-clusters-admin
16:29:19 info deleting clusterroletemplates loft-clusters-creator
16:29:19 info deleting clusterroletemplates loft-impersonator
16:29:19 info deleting clusterroletemplates loft-management-activity-viewer
16:29:19 info deleting clusterroletemplates loft-management-admin
16:29:20 info deleting clusterroletemplates loft-management-authenticated
16:29:20 info deleting clusterroletemplates loft-management-project-admin
16:29:20 info deleting clusterroletemplates loft-management-project-user
16:29:20 info deleting clusterroletemplates loft-management-project-viewer
16:29:20 info deleting clusterroletemplates loft-secrets-admin
16:29:21 info deleting clusterroletemplates loft-secrets-creator
16:29:21 info deleting clusterroletemplates loft-secrets-viewer
16:29:21 info deleting clusterroletemplates loft-teams-admin
16:29:21 info deleting clusterroletemplates loft-teams-creator
16:29:21 info deleting clusterroletemplates loft-teams-viewer
16:29:22 info deleting clusterroletemplates loft-templates-admin
16:29:22 info deleting clusterroletemplates loft-templates-creator
16:29:22 info deleting clusterroletemplates loft-templates-viewer
16:29:22 info deleting clusterroletemplates loft-users-admin
16:29:22 info deleting clusterroletemplates loft-users-creator
16:29:23 info deleting clusterroletemplates loft-users-viewer
16:29:23 debug checking all "clusterroletemplates"
16:29:23 debug checking all "spacetemplates"
16:29:23 info deleting spacetemplates isolated-space
16:29:25 debug checking all "spacetemplates"
16:29:25 debug checking all "apps"
16:29:25 info deleting apps argocd
16:29:25 info deleting apps central-hostpath-mapper
16:29:25 info deleting apps cert-manager
16:29:25 info deleting apps example-bash
16:29:25 info deleting apps example-helm
16:29:25 info deleting apps example-manifests
16:29:25 info deleting apps ingress-nginx
16:29:25 info deleting apps jspolicy
16:29:25 info deleting apps kube-prometheus-stack
16:29:26 info deleting apps opentelemetry-collector
16:29:27 debug checking all "apps"
16:29:27 debug legacy resource "spaceconstraints" not found in discovery, skipping
16:29:27 debug checking all "tasks"
16:29:27 debug checking all "clusterquotas"
16:29:27 debug checking all "projects"
16:29:27 info deleting projects default
16:29:29 debug checking all "projects"
16:29:31 debug checking all "projects"
16:29:33 debug checking all "projects"
16:29:33 debug checking all "runners"
16:29:33 info deleting runners local
16:29:35 debug checking all "runners"
16:29:35 debug checking all "clusters"
16:29:35 info deleting clusters loft-cluster
16:29:37 debug checking all "clusters"
16:29:37 debug checking all "clusteraccesses"
16:29:37 info deleting clusteraccesses loft-admins
16:29:39 debug checking all "clusteraccesses"
16:29:39 debug checking all "networkpeers"
16:29:39 info deleting networkpeers loft-7c7685654c-9sg5x.control-plane
16:29:41 debug checking all "networkpeers"
16:29:41 info deleting networkpeers loft-7c7685654c-9sg5x.local.runner
16:29:43 debug checking all "networkpeers"
16:29:43 debug checking all "teams"
16:29:43 info deleting teams loft-admins
16:29:45 debug checking all "teams"
16:29:47 debug checking all "teams"
16:29:49 debug checking all "teams"
16:29:51 debug checking all "teams"
16:29:53 debug checking all "teams"
16:29:55 debug checking all "teams"
16:29:57 debug checking all "teams"
16:29:59 debug checking all "teams"
16:29:59 debug checking all "users"
16:29:59 info deleting users admin
16:30:01 debug checking all "users"
16:30:01 debug checking all "sharedsecrets"
16:30:01 debug checking all "accesskeys"
16:30:01 info deleting accesskeys loft-control-plane
16:30:01 info deleting accesskeys loft-runner-local-bzbjm
16:30:03 debug checking all "accesskeys"
16:30:03 debug legacy resource "localclusteraccesses" not found in discovery, skipping
16:30:03 debug legacy resource "localteams" not found in discovery, skipping
16:30:03 debug legacy resource "localusers" not found in discovery, skipping
16:30:03 info Uninstalling vCluster Platform...
16:30:03 info Executing command: helm uninstall loft --kube-context kind-v1 --namespace vcluster-platform

16:30:03 done Successfully uninstalled vCluster Platform

16:30:03 info deleting clusterrole "loft-agent-cluster"
16:30:03 info deleting clusterrole "loft-runnner-cluster"
16:30:03 info deleting clusterrole "loft-vcluster-cluster"
16:30:03 info deleting clusterrolebinding "loft-agent-cluster-binding"
16:30:03 info deleting clusterrolebinding "loft-runnner-cluster-binding"
16:30:03 info deleting clusterrolebinding "loft-vcluster-cluster-binding"
16:30:03 info deleting customresourcedefinition accesskeys.storage.loft.sh
16:30:03 info deleting customresourcedefinition apps.storage.loft.sh
16:30:03 info deleting customresourcedefinition clusteraccesses.storage.loft.sh
16:30:03 info deleting customresourcedefinition clusterquotas.storage.loft.sh
16:30:04 info deleting customresourcedefinition clusterroletemplates.storage.loft.sh
16:30:04 info deleting customresourcedefinition clusters.storage.loft.sh
16:30:04 info deleting customresourcedefinition devpodenvironmenttemplates.storage.loft.sh
16:30:04 info deleting customresourcedefinition devpodworkspaceinstances.storage.loft.sh
16:30:04 info deleting customresourcedefinition devpodworkspacetemplates.storage.loft.sh
16:30:04 info deleting customresourcedefinition networkpeers.storage.loft.sh
16:30:04 info deleting customresourcedefinition projects.storage.loft.sh
16:30:04 info deleting customresourcedefinition runners.storage.loft.sh
16:30:04 info deleting customresourcedefinition sharedsecrets.storage.loft.sh
16:30:04 info deleting customresourcedefinition spaceinstances.storage.loft.sh
16:30:04 info deleting customresourcedefinition spacetemplates.storage.loft.sh
16:30:05 info deleting customresourcedefinition tasks.storage.loft.sh
16:30:05 info deleting customresourcedefinition teams.storage.loft.sh
16:30:05 info deleting customresourcedefinition users.storage.loft.sh
16:30:05 info deleting customresourcedefinition virtualclusterinstances.storage.loft.sh
16:30:05 info deleting customresourcedefinition virtualclustertemplates.storage.loft.sh
16:30:06 info deleting namespace "vcluster-platform"

vcluster platform destroy (no args) on v3.4.9:

16:27:18 debug checking all "virtualclusterinstances"
16:27:18 debug checking all "virtualclusters"
16:27:18 debug checking all "devpodworkspaceinstances"
16:27:18 debug checking all "spaceinstances"
16:27:18 debug checking all "virtualclustertemplates"
16:27:18 info deleting virtualclustertemplates vcluster-pro-isolated-template
16:27:18 info deleting virtualclustertemplates vcluster-pro-template
16:27:20 debug checking all "virtualclustertemplates"
16:27:20 info resource "devpodenvironmenttemplates" not found in discovery, skipping
16:27:20 debug checking all "devpodworkspacetemplates"
16:27:20 info deleting devpodworkspacetemplates aws
16:27:20 info deleting devpodworkspacetemplates azure
16:27:20 info deleting devpodworkspacetemplates digitalocean
16:27:20 info deleting devpodworkspacetemplates ecs
16:27:20 info deleting devpodworkspacetemplates gcloud
16:27:20 info deleting devpodworkspacetemplates kubernetes
16:27:22 debug checking all "devpodworkspacetemplates"
16:27:22 debug checking all "clusterroletemplates"
16:27:22 info deleting clusterroletemplates loft-apps-admin
16:27:22 info deleting clusterroletemplates loft-apps-creator
16:27:22 info deleting clusterroletemplates loft-apps-viewer
16:27:22 info deleting clusterroletemplates loft-cluster-admin
16:27:22 info deleting clusterroletemplates loft-cluster-authenticated
16:27:22 info deleting clusterroletemplates loft-cluster-extra-space-admin-rules
16:27:22 info deleting clusterroletemplates loft-cluster-extra-space-viewer-rules
16:27:22 info deleting clusterroletemplates loft-cluster-space-admin
16:27:22 info deleting clusterroletemplates loft-cluster-space-viewer
16:27:23 info deleting clusterroletemplates loft-clusters-admin
16:27:23 info deleting clusterroletemplates loft-clusters-creator
16:27:23 info deleting clusterroletemplates loft-impersonator
16:27:23 info deleting clusterroletemplates loft-management-activity-viewer
16:27:23 info deleting clusterroletemplates loft-management-admin
16:27:24 info deleting clusterroletemplates loft-management-authenticated
16:27:24 info deleting clusterroletemplates loft-management-project-admin
16:27:24 info deleting clusterroletemplates loft-management-project-user
16:27:24 info deleting clusterroletemplates loft-management-project-viewer
16:27:24 info deleting clusterroletemplates loft-secrets-admin
16:27:25 info deleting clusterroletemplates loft-secrets-creator
16:27:25 info deleting clusterroletemplates loft-secrets-viewer
16:27:25 info deleting clusterroletemplates loft-teams-admin
16:27:25 info deleting clusterroletemplates loft-teams-creator
16:27:25 info deleting clusterroletemplates loft-teams-viewer
16:27:26 info deleting clusterroletemplates loft-templates-admin
16:27:26 info deleting clusterroletemplates loft-templates-creator
16:27:26 info deleting clusterroletemplates loft-templates-viewer
16:27:26 info deleting clusterroletemplates loft-users-admin
16:27:26 info deleting clusterroletemplates loft-users-creator
16:27:27 info deleting clusterroletemplates loft-users-viewer
16:27:27 debug checking all "clusterroletemplates"
16:27:27 debug checking all "spacetemplates"
16:27:27 info deleting spacetemplates isolated-space
16:27:29 debug checking all "spacetemplates"
16:27:29 debug checking all "apps"
16:27:29 info deleting apps argocd
16:27:29 info deleting apps central-hostpath-mapper
16:27:29 info deleting apps cert-manager
16:27:29 info deleting apps example-bash
16:27:29 info deleting apps example-helm
16:27:29 info deleting apps example-manifests
16:27:29 info deleting apps ingress-nginx
16:27:29 info deleting apps jspolicy
16:27:29 info deleting apps kube-prometheus-stack
16:27:30 info deleting apps opentelemetry-collector
16:27:31 debug checking all "apps"
16:27:31 debug checking all "spaceconstraints"
16:27:31 debug checking all "tasks"
16:27:31 debug checking all "clusterquotas"
16:27:31 debug checking all "projects"
16:27:31 info deleting projects default
16:27:33 debug checking all "projects"
16:27:35 debug checking all "projects"
16:27:37 debug checking all "projects"
16:27:37 debug checking all "runners"
16:27:37 info deleting runners local
16:27:39 debug checking all "runners"
16:27:39 debug checking all "clusters"
16:27:39 info deleting clusters loft-cluster
16:27:41 debug checking all "clusters"
16:27:41 debug checking all "clusteraccesses"
16:27:41 info deleting clusteraccesses loft-admins
16:27:43 debug checking all "clusteraccesses"
16:27:43 debug checking all "networkpeers"
16:27:43 info deleting networkpeers loft-b67bf7475-r7knc.control-plane
16:27:45 debug checking all "networkpeers"
16:27:45 debug checking all "teams"
16:27:45 info deleting teams loft-admins
16:27:47 debug checking all "teams"
16:27:49 debug checking all "teams"
16:27:51 debug checking all "teams"
16:27:53 debug checking all "teams"
16:27:55 debug checking all "teams"
16:27:57 debug checking all "teams"
16:27:59 debug checking all "teams"
16:27:59 debug checking all "users"
16:27:59 info deleting users admin
16:28:01 debug checking all "users"
16:28:01 debug checking all "sharedsecrets"
16:28:01 debug checking all "accesskeys"
16:28:01 info deleting accesskeys loft-agent-loft-cluster-47287
16:28:01 info deleting accesskeys loft-control-plane
16:28:01 info deleting accesskeys loft-runner-local-xsblh
16:28:03 debug checking all "accesskeys"
16:28:03 debug checking all "localclusteraccesses"
16:28:03 debug checking all "localteams"
16:28:03 debug checking all "localusers"
16:28:03 info Uninstalling vCluster Platform...
16:28:03 info Executing command: helm uninstall loft --kube-context kind-v3 --namespace vcluster-platform

16:28:03 done Successfully uninstalled vCluster Platform

16:28:03 info deleting clusterrole "loft-agent-cluster"
16:28:03 info deleting clusterrole "loft-runnner-cluster"
16:28:03 info deleting clusterrole "loft-vcluster-cluster"
16:28:03 info deleting clusterrolebinding "loft-agent-cluster-binding"
16:28:03 info deleting clusterrolebinding "loft-runnner-cluster-binding"
16:28:03 info deleting clusterrolebinding "loft-vcluster-cluster-binding"
16:28:03 info deleting customresourcedefinition accesskeys.storage.loft.sh
16:28:03 info deleting customresourcedefinition apps.storage.loft.sh
16:28:03 info deleting customresourcedefinition clusteraccesses.storage.loft.sh
16:28:03 info deleting customresourcedefinition clusterquotas.storage.loft.sh
16:28:03 info deleting customresourcedefinition clusterroletemplates.storage.loft.sh
16:28:03 info deleting customresourcedefinition clusters.storage.loft.sh
16:28:03 info deleting customresourcedefinition devpodworkspaceinstances.storage.loft.sh
16:28:03 info deleting customresourcedefinition devpodworkspacetemplates.storage.loft.sh
16:28:04 info deleting customresourcedefinition localclusteraccesses.storage.loft.sh
16:28:04 info deleting customresourcedefinition localteams.storage.loft.sh
16:28:04 info deleting customresourcedefinition localusers.storage.loft.sh
16:28:04 info deleting customresourcedefinition networkpeers.storage.loft.sh
16:28:04 info deleting customresourcedefinition projects.storage.loft.sh
16:28:04 info deleting customresourcedefinition runners.storage.loft.sh
16:28:04 info deleting customresourcedefinition sharedsecrets.storage.loft.sh
16:28:05 info deleting customresourcedefinition spaceconstraints.storage.loft.sh
16:28:05 info deleting customresourcedefinition spaceinstances.storage.loft.sh
16:28:05 info deleting customresourcedefinition spacetemplates.storage.loft.sh
16:28:05 info deleting customresourcedefinition tasks.storage.loft.sh
16:28:05 info deleting customresourcedefinition teams.storage.loft.sh
16:28:06 info deleting customresourcedefinition users.storage.loft.sh
16:28:06 info deleting customresourcedefinition virtualclusterinstances.storage.loft.sh
16:28:06 info deleting customresourcedefinition virtualclusters.storage.loft.sh
16:28:06 info deleting customresourcedefinition virtualclustertemplates.storage.loft.sh
16:28:07 info deleting namespace "vcluster-platform"
netlify[bot] commented 2 weeks ago

Deploy Preview for vcluster-docs canceled.

Built without sensitive environment variables

Name Link
Latest commit 81fa5e42845a62d400126b0ed9be993e3b4c2cfd
Latest deploy log https://app.netlify.com/sites/vcluster-docs/deploys/673f09a5de23b90008bed865
rohantmp commented 2 weeks ago

I'll move it into a destroy folder and move some of the common options out into clihelper

rohantmp commented 2 weeks ago

~As I was looking into refactoring, I'm starting to think it makes sense to have start and destroy together. It's is like having install and uninstall together. There are lots of common things and cli argument sanity checking.~

~Feels a bit awkward to put sanity checking of args that hold the same set of requirements and clients in a destroy package.~ That said, I could leave that bit in start and just import it from destroy.

rohantmp commented 2 weeks ago

Ready to review

rohantmp commented 23 hours ago

Oh wait, there's more capitlization pointed out I missed

rohantmp commented 23 hours ago

Done, fixed capitalization and "loft" mentions in the touched files