[Rancher2] Document vSphere Out Of Tree on an RKE Local cluster

[sowmyav27]

Related issue: https://github.com/rancher/rancher/issues/31839

[bmdepesa]

Most of the existing Rancher documentation is the same regarding vSphere versions, checks between steps, etc.. the only big difference is how we install the charts.

Fresh Installs

Set the external cloud provider on the cluster yaml and run rke up

cloud_provider:
    name: external

Add the charts repo to helm and update

helm repo add rancher-charts http://charts.rancher.io/
helm repo update

Install the charts by passing the vCenter information to the chart options. Users can view the values.yaml for any advanced configuration. If users wish to set the configuration secrets/configmaps themselves rather than having the chart generate them, they can follow the official documentation and set the generate flags on the chart install to false: https://vsphere-csi-driver.sigs.k8s.io/driver-deployment/prerequisites.html https://vsphere-csi-driver.sigs.k8s.io/driver-deployment/installation.html

helm install vsphere-cpi rancher-charts/rancher-vsphere-cpi -n kube-system \
--set vCenter.host="" \
--set vCenter.datacenters="" \
--set vCenter.username="" \
--set vCenter.password="" \
--set vCenter.credentialsSecret.generate=false # set to false if configuring manually (default true) 

helm install vsphere-csi rancher-charts/rancher-vsphere-csi -n kube-system \
--set vCenter.host="" \
--set vCenter.datacenters="" \
--set vCenter.username="" \
--set vCenter.password=""\
--set vCenter.clusterId="" \
--set storageClass.datastoreURL="" \
--set vCenter.configSecret.generate=true \ # set to false if configuring manually (default true)
--set csiController.csiResizer.enabled=false #defaults to false, can be enabled if on an environment that supports resizing

Migration

Steps for migration are also similar

• Install the charts as in the fresh install, but also pass the migration flag to the CSI chart

  helm install vsphere-csi rancher-charts/rancher-vsphere-csi -n kube-system \
  --set vCenter.host="" \
  --set vCenter.datacenters="" \
  --set vCenter.username="" \
  --set vCenter.password=""\
  --set vCenter.clusterId="" \
  --set storageClass.datastoreURL="" \
  --set csiMigration.enabled=true

• Edit the cluster to enable migration and draining

  kube-controller:
  extra_args:
  feature-gates: 'CSIMigration=true,CSIMigrationvSphere=true'
  kubelet:
  extra_args:
  feature-gates: 'CSIMigration=true,CSIMigrationvSphere=true'
  ..
  upgrade_strategy:
  max_unavailable_controlplane: '1'
  max_unavailable_worker: 1
  node_drain_input:
    delete_local_data: true
    force: true

  rke up

  Then perform the same validations as in the Rancher docs to see that the volumes have been migrated

[Tejeev]

@bmdepesa Just checking but those Migration steps are for in tree -> out of tree, right?

[bmdepesa]

@Tejeev yes from in-tree -> out-of-tree

Initial configuration has the in-tree cloud provider information defined in the cluster yaml.

The migration steps are the same as outlined in Rancher docs, the only difference is in how we update the charts and edit the cluster -- taint the nodes manually or via the script provided in the Rancher docs, then install the charts w/ the CSI migration flag enabled, and update the cluster yaml and update the cluster with RKE.

[obiordu]

A fresh install:

To make this work for my environment, i needed to check that the providerID is set after deploying the CPI using: kubectl get nodes -o json | jq '.items[]|[.metadata.name, .spec.providerID, .status.nodeInfo.systemUUID]' OR kubectl describe nodes | grep "ProviderID"

If the output for providerID is null, then the kubernetes node object needs to be patched for all the nodes individually before deploying the CSI for the storageClass, PV and PVC to work/bind: kubectl patch node -p "{"\"spec\":{\"providerID\":\"vsphere://nnnnnnnn-nnnn-nnnn-nnnn-nnnnnnnnnnnn\"}}"

https://github.com/rancher/rancher/issues/18949#issuecomment-648256475

[github-actions[bot]]

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 90 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 30 days. Thank you for your contributions.