failed to get shared datastores in kubernetes cluster

[MKITConsulting]

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug

What happened:

The attempt to provision a new PVC leads to the following error messages on attempt to deploy hashicorp vault helm chart (works on another cluster though):

2024-10-09T12:12:14.699271218Z W1009 12:12:14.699128       1 controller.go:934] Retrying syncing claim "c18c109b-9b36-4405-9834-5a5a09198776", failure 9 
2024-10-09T12:12:14.699280063Z E1009 12:12:14.699157       1 controller.go:957] error syncing claim "c18c109b-9b36-4405-9834-5a5a09198776": failed to provision volume with StorageClass "vsphere-csi-sc": rpc error: code = Internal desc = failed to get shared datastores in kubernetes cluster. Error: ServerFaultCode: The object 'vim.VirtualMachine:vm-567408' has already been deleted or has not been completely created

What you expected to happen:

PVC gets created

How to reproduce it (as minimally and precisely as possible):

Latest Hashicorp Vault Helm Chart deployment.

Anything else we need to know?:

Environment:

(taken from Helm release values):

csiController:
  csiResizer:
    enabled: false
  image:
    csiAttacher:
      repository: rancher/mirrored-sig-storage-csi-attacher
      tag: v4.2.0
    csiProvisioner:
      repository: rancher/mirrored-sig-storage-csi-provisioner
      tag: v3.4.0
    csiResizer:
      repository: rancher/mirrored-sig-storage-csi-resizer
      tag: v1.7.0
    livenessProbe:
      repository: rancher/mirrored-sig-storage-livenessprobe
      tag: v2.9.0
    repository: rancher/mirrored-cloud-provider-vsphere-csi-release-driver
    tag: v3.0.1
    vsphereSyncer:
      repository: rancher/mirrored-cloud-provider-vsphere-csi-release-syncer
      tag: v3.0.1