Secrets not syncing when genericSync exports a secret

[eryalito]

What happened?

I created a SA and token on bootstrap using the experimental.deploy.vcluster.manifests feature. I configured the experimental.genericSync.export to sync the resulting secret from the virtual cluster into the physical one.

```yaml experimental: genericSync: export: - apiVersion: v1 kind: Secret selector: labelSelector: kubernetes.io/service-account.name: vcluster-admin patches: - op: replace path: .type value: Opaque - op: replace path: .metadata.name value: vc-admin-token deploy: vcluster: manifests: |- --- apiVersion: v1 kind: ServiceAccount metadata: name: vcluster-admin namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: vcluster-admin roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: vcluster-admin namespace: kube-system --- apiVersion: v1 kind: Secret metadata: name: vcluster-admin namespace: kube-system labels: kubernetes.io/service-account.name: vcluster-admin annotations: kubernetes.io/service-account.name: vcluster-admin type: kubernetes.io/service-account-token ```

This works as expected. But after that, when creating any kind of pod that mounts a secret on the virtual cluster (I tested nginx ingress controller) the pod is correctly synced to the physical cluster, but the secret is stuck on a loop of creating and deleting. Logs:

release-name-57588f9fcf-vlskk syncer 2024-09-03 16:01:49    INFO    secret.ingress-nginx-admission  secrets/syncer.go:111   create physical secret test-vcluster/ingress-nginx-admission-x-ingress-nginx-x-release-name {"component": "vcluster"}
release-name-57588f9fcf-vlskk syncer 2024-09-03 16:01:49    INFO    secret//GenericExport.ingress-nginx-admission   syncer/syncer.go:136    delete physical Secret test-vcluster/ingress-nginx-admission-x-ingress-nginx-x-release-name, because it is not used anymore {"component": "vcluster"}

What did you expect to happen?

Both the manually synced secret and the one needed by the pod are synced and not deleted from the physical cluster.

How can we reproduce it (as minimally and precisely as possible)?

It can be reproduced by enabling/disabling the genericSync export of the secret

Anything else we need to know?

I tried some workarounds like enabling all secrets sync, but it does not stop the loop, it just starts creating and deleting all of the secrets

sync:
  toHost:
    secrets:
      all: true

Host cluster Kubernetes version

```console $ kubectl version Server Version: version.Info{Major:"1", Minor:"30", GitVersion:"v1.30.0", GitCommit:"7c48c2bd72b9bf5c44d21d7338cc7bea77d0ad2a", GitTreeState:"clean", BuildDate:"2024-04-17T17:27:03Z", GoVersion:"go1.22.2", Compiler:"gc", Platform:"linux/amd64"} ```

vcluster version

```console $ vcluster --version vcluster version 0.20.0 ```

VCluster Config

``` controlPlane: distro: k8s: enabled: true version: "v1.29.0" backingStore: etcd: deploy: enabled: true statefulSet: highAvailability: replicas: 3 statefulSet: highAvailability: replicas: 3 image: repository: "loft-sh/vcluster-oss" experimental: genericSync: export: - apiVersion: v1 kind: Secret selector: labelSelector: kubernetes.io/service-account.name: vcluster-admin patches: - op: replace path: .type value: Opaque - op: replace path: .metadata.name value: vc-admin-token deploy: vcluster: manifests: |- --- apiVersion: v1 kind: ServiceAccount metadata: name: vcluster-admin namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: vcluster-admin roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: vcluster-admin namespace: kube-system --- apiVersion: v1 kind: Secret metadata: name: vcluster-admin namespace: kube-system labels: kubernetes.io/service-account.name: vcluster-admin annotations: kubernetes.io/service-account.name: vcluster-admin type: kubernetes.io/service-account-token ```

[facchettos]

hi @eryalito ! I have fixed the issue in #2126 , it will be released in the next minor version

[eryalito]

Thank you @facchettos !