harvester / harvester

Open source hyperconverged infrastructure (HCI) software
https://harvesterhci.io/
Apache License 2.0
3.86k stars 325 forks source link

[ENHANCEMENT] let user could setup `HostStorageClass` with helm option #4045

Closed Vicente-Cheng closed 2 weeks ago

Vicente-Cheng commented 1 year ago

Is your enhancement related to a problem? Please describe. The environment variable HostStorageClass could be set for the default StorageClass associated with the bare-metal harvester cluster of the harvester CSI driver. We may expose this variable to set it up when deploying the downstream cluster.

Describe the solution you'd like Expose the HostStorageClass with the helm chart

Describe alternatives you've considered None

Additional context Reference for this variable: https://github.com/harvester/harvester-csi-driver/blob/master/main.go#L41

harvesterhci-io-github-bot commented 11 months ago

Pre Ready-For-Testing Checklist

~* [ ] If labeled: require/HEP Has the Harvester Enhancement Proposal PR submitted? The HEP PR is at:~

Test Plan:

  1. create Harvester cluster
  2. create a test StorageClass
  3. create downstream cluster
  4. upgrade CSI driver from v0.1.18 to v0.1.19 You can manually use my repo for it (REPO: https://github.com/Vicente-Cheng/rancher-charts, branch: bump-harvester-csi-driver-to-v2.8) I created branch for rancher v2.8, so you need to check your downstream cluster created with rancher v2.8
  5. when you upgrade, choose the customize the helm option before install
  6. Then, choose the edit yaml, you will find the parameter called: hostStorageClass
  7. edit it to point to the host cluster SC created in step 2.
  8. upgrade
  9. After upgrade, create PVC with default harvester cluster SC (on downstream cluster)
  10. Check the host cluster volume. It should use the SC we assigned in step 7.

~* [ ] Is there a workaround for the issue? If so, where is it documented? The workaround is at:~

~ [ ] Have the backend code been merged (harvester, harvester-installer, etc) (including `backport-needed/`)? The PR is at:~

~* [ ] If labeled: area/ui Has the UI issue filed or ready to be merged? The UI issue/PR is at:~

~* [ ] If NOT labeled: not-require/test-plan Has the e2e test plan been merged? Have QAs agreed on the automation test case? If only test case skeleton w/o implementation, have you created an implementation issue?

~* [ ] If the fix introduces the code for backward compatibility Has a separate issue been filed with the label release/obsolete-compatibility? The compatibility issue is filed at:~

harvesterhci-io-github-bot commented 11 months ago

Automation e2e test issue: harvester/tests#1007

irishgordo commented 8 months ago

Hi, @Vicente-Cheng - I've been working through testing this but had some validation questions.

Is it possible to validate this by "re-deploying" the harvester-csi-driver App from Rancher UI w/ "Edit YAML" and on the helm options:

As I've been trying with separate loadouts of "Edit YAML" either taking the shape of the following.

Edited YAML Example A:

cloudConfig:
  hostPath: /var/lib/rancher/rke2/etc/config-files/
  secretName: ''
fullnameOverride: ''
global:
  cattle:
    systemProjectId: p-cgbv4
  clusterCIDR: 10.42.0.0/16
  clusterCIDRv4: 10.42.0.0/16
  clusterDNS: 10.43.0.10
  clusterDomain: cluster.local
  rke2DataDir: /var/lib/rancher/rke2
  serviceCIDR: 10.43.0.0/16
image:
  csi:
    attacher:
      repository: rancher/mirrored-longhornio-csi-attacher
      tag: v3.2.1
    nodeDriverRegistrar:
      repository: rancher/mirrored-longhornio-csi-node-driver-registrar
      tag: v2.3.0
    provisioner:
      repository: rancher/mirrored-longhornio-csi-provisioner
      tag: v2.1.2
    resizer:
      repository: rancher/mirrored-longhornio-csi-resizer
      tag: v1.2.0
  harvester:
    csiDriver:
      repository: rancher/harvester-csi-driver
      tag: v0.1.5
      args:
        - '--host-storage-class=extrasc'
  pullPolicy: IfNotPresent
kubeletRootDir: /var/lib/kubelet
nameOverride: ''
nodeSelector:
  kubernetes.io/os: linux
replicasCount: 3
tolerations:
  - effect: NoSchedule
    key: kubevirt.io/drain
    operator: Exists
  - effect: NoSchedule
    key: node-role.kubernetes.io/control-plane
    operator: Equal
  - effect: NoExecute
    key: node-role.kubernetes.io/etcd
    operator: Equal
  - effect: NoSchedule
    key: cattle.io/os
    operator: Equal
    value: linux

Edited YAML Example B:

cloudConfig:
  hostPath: /var/lib/rancher/rke2/etc/config-files/
  secretName: ''
fullnameOverride: ''
global:
  cattle:
    systemProjectId: p-cgbv4
  clusterCIDR: 10.42.0.0/16
  clusterCIDRv4: 10.42.0.0/16
  clusterDNS: 10.43.0.10
  clusterDomain: cluster.local
  rke2DataDir: /var/lib/rancher/rke2
  serviceCIDR: 10.43.0.0/16
image:
  csi:
    attacher:
      repository: rancher/mirrored-longhornio-csi-attacher
      tag: v3.2.1
    nodeDriverRegistrar:
      repository: rancher/mirrored-longhornio-csi-node-driver-registrar
      tag: v2.3.0
    provisioner:
      repository: rancher/mirrored-longhornio-csi-provisioner
      tag: v2.1.2
    resizer:
      repository: rancher/mirrored-longhornio-csi-resizer
      tag: v1.2.0
  harvester:
    csiDriver:
      repository: rancher/harvester-csi-driver
      tag: v0.1.5
      env:
        - name: HOST_STORAGE_CLASS
          value: extrasc
  pullPolicy: IfNotPresent
kubeletRootDir: /var/lib/kubelet
nameOverride: ''
nodeSelector:
  kubernetes.io/os: linux
replicasCount: 3
tolerations:
  - effect: NoSchedule
    key: kubevirt.io/drain
    operator: Exists
  - effect: NoSchedule
    key: node-role.kubernetes.io/control-plane
    operator: Equal
  - effect: NoExecute
    key: node-role.kubernetes.io/etcd
    operator: Equal
  - effect: NoSchedule
    key: cattle.io/os
    operator: Equal
    value: linux

With either edited yaml loadouts on:

I'm not able to build a PVC for the RKE2 (Kubernetes Version: v1.27.10 +rke2r1) cluster that when selecting harvester will utilize the extrasc storage class, it's still all defaulting to the default storage-class.

Artifacts: Screenshot from 2024-02-13 16-00-48 Screenshot from 2024-02-13 15-59-58 Screenshot from 2024-02-13 15-56-06 Screenshot from 2024-02-13 15-44-43

Vicente-Cheng commented 8 months ago

Hi @irishgordo, I thought I needed some clarification here.

I move this one back to the Design to clarify again. Thanks for the testing!

Vicente-Cheng commented 3 weeks ago

Update the test plan. now we can use helm option to setup HostStorageClass

irishgordo commented 2 weeks ago

@Vicente-Cheng thanks for updating the test plan :smile: :+1: Vallidation Question:

Or is that not needed to do the manual chart upgrade?

Vicente-Cheng commented 2 weeks ago

The PR is already merged into dev-v2.8 branch on https://github.com/rancher/charts.git

So, you can try to use the repo: https://github.com/rancher/charts.git, branch: dev-v2.8 You can search harvester-csi (by filter) on the Apps->Charts It should have an extra one

irishgordo commented 2 weeks ago

Thanks for that mention @Vicente-Cheng This all looks great on v1.4.0-rc4 w/ Rancher v2.9.2 (vCluster, k3s v1.29) Upgrade of CSI 0.1.18 -> 0.1.20 w/ hostStorageClass

Supporting Screengrabs: Screenshot from 2024-10-23 22-48-18 Screenshot from 2024-10-23 22-47-59 Screenshot from 2024-10-23 22-47-02 Screenshot from 2024-10-23 22-44-04 Screenshot from 2024-10-23 22-43-30 Screenshot from 2024-10-23 22-42-37 Screenshot from 2024-10-23 22-41-27 Screenshot from 2024-10-23 22-40-58 Screenshot from 2024-10-23 22-40-12 Screenshot from 2024-10-23 22-37-43 Screenshot from 2024-10-23 22-35-46

This all looks good :smile: :+1: -> I'll go ahead and close this out :+1: