loft-sh / vcluster

vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
https://www.vcluster.com
Apache License 2.0
6.25k stars 395 forks source link

Vcluster deletion using cli is not working as expected with VCluster argocd application #1917

Open adamissaoui opened 1 month ago

adamissaoui commented 1 month ago

What happened?

when creating vcluster using cli, the deletion works fine however when creating the vcluster using argocd application the deletion is not works.

only if i delete the argocd application the vcluster got deleted.

What did you expect to happen?

The vcluster delete should delete any vcluster that can be listed through cli

How can we reproduce it (as minimally and precisely as possible)?

  1. create vcluster using argocd application :
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: vc-1
  namespace: argocd
spec:
  project: vcluster
  destination:
    server: https://kubernetes.default.svc/
    namespace: vcluster-vc-1
  syncPolicy:
    automated:
      selfHeal: true
    syncOptions:
    - CreateNamespace=true
  source:
    path: ''
    repoURL: 'https://charts.loft.sh/'
    targetRevision: 0.19.6
    chart: vcluster
  1. vcluster list should return the vcluster vc-1 :

vcluster list | grep vc-1

Output :

vc-1 | arn:aws:eks:us-east-1:123456789:cluster/VCLUSTER | vcluster-vc-1 | Running | 0.19.6 | | 2024-07-09 10:39:54 +0100 IST | 24h57m51s | OSS

  1. try to use vcluster using : vcluster delete vc-1 --debug
20:55:17 debug Error creating pro client: couldn't find vCluster.Pro config: please make sure to run 'vcluster login' to connect to an existing instance or 'vcluster pro start' to deploy a new instance
20:55:18 info Delete vcluster vc-1...
20:55:19 debug Delete helm chart with helm delete vc-1 --namespace vcluster-vc-1 --kubeconfig /var/folders/t7/x7xdvdh90ms9_rcynjbh00rr0000gs/T/2513715556 --repository-config=''
20:55:20 fatal release 'vc-1' was not found in namespace 'vcluster-vc-1'

Anything else we need to know?

No response

Host cluster Kubernetes version

```console $ kubectl version kubectl version Client Version: v1.28.3 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.29.4-eks-036c24b ```

vcluster version

```console $ vcluster --version vcluster version 0.19.5 ```

VCluster Config

``` # My vcluster.yaml / values.yaml here ```
machecazzon commented 1 month ago

Hi, I think this might be related to argocd as it does not use helm but helm template and applying manifests what in the end is not a helm release (helm release version secret will not exist)

deniseschannon commented 1 month ago

@machecazzon is correct. Since deploying with ArgoCD uses helm template, the virtual cluster is deployed with manifests instead of a helm release. The vcluster CLI currently assumes that the virtual clusters were created with helm (not with manifests) and try to delete using helm, which relies on the helm release secret.

We've added an enhancement into our backlog in order to improve the experience.