search

vmware-archive / kubernetes-archived

This repository is archived. Please file in-tree vSphere Cloud Provider issues at https://github.com/kubernetes/kubernetes/issues . CSI Driver for vSphere is available at https://github.com/kubernetes/cloud-provider-vsphere
Apache License 2.0
46 stars 31 forks source link

Provision vsphere volumes as per zone and attach labels #517

Open skarthiksrinivas opened 5 years ago

skarthiksrinivas commented 5 years ago

What type of PR is this? /kind bug

What this PR does / why we need it: Currently vsphere cloud provider (VCP) insists on provisioning a volume only on a globally shared datastore. Hence, in a zoned environment, even in presence of a shared datastore within a specific zone, the volume provisioning can fail if that datastore is not shared across all the zones hosting kubernetes nodes. This change fixes this issue by considering the zone information provided in allowedTopologies for selection of the datastore. If allowedTopologies is not provided, the current behaviour is retained as-is. Further, after a volume is created on vSphere, the zone label does not get applied on it today. Without this information on the volume, a pod that uses this volume could get scheduled to a different zone where this volume is not visible/accessible. This issue can be solved by just labeling the volume with the zone information. This PR computes the zone of a volume in the PersistentVolumeLabel admission controller since vsphere is still an in-tree cloud provider and the out-of-tree plugin in still the works. Which issue(s) this PR fixes: Fixes #67703

Does this PR introduce a user-facing change?: Yes

This change ensures that volumes get provisioned based on the zone information provided in allowedTopologies and gets appropriate label.

Storage class spec:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: fastpolicy1
provisioner: kubernetes.io/vsphere-volume
parameters:
    diskformat: zeroedthick
    storagePolicyName: vSAN Default Storage Policy
allowedTopologies:
- matchLabelExpressions:
  - key: failure-domain.beta.kubernetes.io/zone
    values:
    - zone1

After PV creation:
[container]:/pks> kubectl get pv --show-labels
NAME   CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS   CLAIM   STORAGECLASS   REASON  
 AGE   LABELS
pvc-ba4b80cc-100f-11e9-9143-005056804cc9   1Gi   RWO   Delete   Bound   default/pvcsc-1-policy 
  fastpolicy1   2s   failure-domain.beta.kubernetes.io/region=EMEA,failure-domain.beta.kubernetes.io/zone=zone1

New zone tests:
http://blr-dbc505.eng.vmware.com/sandeepsunny/zone-tests.log
Ran 15 of 2135 Specs in 1203.660 seconds
SUCCESS! -- 15 Passed | 0 Failed | 0 Pending | 2120 Skipped PASS

Regression test:
http://cna-storage-jenkins.eng.vmware.com/job/Run-k8s-test-latest/76/consoleFull
Ran 35 of 2119 Specs in 4579.419 seconds
FAIL! -- 34 Passed | 1 Failed | 0 Pending | 2084 Skipped --- FAIL: TestE2E (4579.74s)
The single failure is a test issue.
skarthiksrinivas commented 5 years ago

Addressed Subbu's comments.