Open jsoule6 opened 1 year ago
The only thing we can think of that might be causing this is that the host that the Control Plane node is on does not have access to the same datastore that the Worker nodes all do.
This is the reason the driver is not able to find the shared accessible datastore for all nodes. We should have a datastore accessible to all nodes in the cluster including control plane nodes.
We encountered the same issue, when we deploy our workers to a single Vsphere cluster using VSAN storage it works fine, but when we split the workers to be deployed across 2 vsphere clusters, each cluster with their own VSAN storage we get the same error when creating the PVC.
Is there a way in this setup for the PVC to only go to 1 of the vsphere clusters and its VSAN storage?
in our setup its now possible for the 2 vsphere clusters to have shared storage, they each have their own VSAN Storage
@sba30 you can define topology on the nodes, and utilize volume topology feature to provision volume on specific vSphere cluster. https://docs.vmware.com/en/VMware-vSphere-Container-Storage-Plug-in/3.0/vmware-vsphere-csp-getting-started/GUID-162E7582-723B-4A0F-A937-3ACE82EAFD31.html
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
/remove-lifecycle rotten
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
Is there a way in this setup for the PVC to only go to 1 of the vsphere clusters and its VSAN storage?
We have a similar request for CSI driver to have PVC only on a subset of worker nodes. In our use cases, control-plane and worker nodes are in different vSphere datacenters, and we only need volumns on the worker nodes.
@sba30 you can define topology on the nodes, and utilize volume topology feature to provision volume on specific vSphere cluster. https://docs.vmware.com/en/VMware-vSphere-Container-Storage-Plug-in/3.0/vmware-vsphere-csp-getting-started/GUID-162E7582-723B-4A0F-A937-3ACE82EAFD31.html
It is mentioned that "If you already use vSphere Container Storage Plug-in to run applications, but haven't used the topology feature, you must re-create the entire cluster and delete any existing PVCs in the system to be able to use the topology feature.". As we are looking for migrating existing product clusters to the aforementioned setup, re-creating the entire cluster is likely not an option for us.
@xing-yang @divyenpatel Could you please help take a look at this issue? Thank you!
To add more details, we are looking for support to exclude certain nodes in the clusters from CSI driver and PVs won't be attached to those ever.
As for our use cases, we plan to add new nodes in one vSphere datacenter, to an existing cluster where all the nodes are in another vSpehre datacenter and the CSI is functional.
Learned that this change https://github.com/kubernetes-sigs/vsphere-csi-driver/pull/2412 by @gn will allow day2 topology operation, we do not need to recreate the whole cluster.
@YuanPeterGao
Learned that this change https://github.com/kubernetes-sigs/vsphere-csi-driver/pull/2412 by @gn will allow day2 topology operation, we do not need to recreate the whole cluster.
@YuanPeterGao
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
/remove-lifecycle rotten
Issue still exists. I just hit it today...
Environment:
vSphere Client Version: 8.0.2.00300 CSI Driver: v4.0.0_vmware.1 Kubernetes: v1.28.7+vmware.1
# kubectl version --output=yaml
clientVersion:
buildDate: "2023-06-27708:59: 392"
compiler: gc
gitCommit: 094a5d36c0a04fec8700031caf9c63cec5fda2c8
gitTreeState: clean
gitVersion: v1.27.2+vmware.2
goVersion: go1.20.4
major: "1"
minor: "27"
platform: linux/amd64
kustomizeVersion: v5.0.1
serverVersion:
buildDate: "2024-02-19T11:29: 19"
compiler: gc
gitCommit: 1630090845297a4603596750ce2833d35761bfe
gitTreeState: clean
gitVersion: v1.28.7+vmware.1
goVersion: go1.21.7
major: "1"
minor: "28"
platform: linux/amd64
Is this a BUG REPORT or FEATURE REQUEST?:
What happened:
Installed the CSI driver and Cloud Controller Manager on a K8s cluster running on vSphere VMs. Everything installed well and it is connecting to vSphere and successfully getting node information. However, when we try to deploy a pod on the cluster with the new Storage Class that we created, we are getting the following error:
failed to get shared datastores in kubernetes cluster. Error: no shared datastores found for nodeVm.
We have a single vCenter and are not using a topology aware setup. We have checked the permissions on the vSphere side for the account that we are using and all looks good. The only thing we can think of that might be causing this is that the host that the Control Plane node is on does not have access to the same datastore that the Worker nodes all do. We tried applying and using a Storage Policy as well, but with the same result.
Is it a requirement that all nodes including the Control Plane have access to at least one Shared datastore?
What you expected to happen:
I would expect the PVC to be created
How to reproduce it (as minimally and precisely as possible):
Create a K8s cluster with CSI 2.5.4 and the other versions mentioned above.
Anything else we need to know?:
Environment: Using the following versions:
vSphere: 6.7 Update 3 Kubernetes: 1.21 Cloud Controller Manager: 1.21 CSI Driver: 2.5.4