Unable to deploy storageClass: democratic-csi-iscsi-controller

[lknite]

What happened?

Truenas provides kubernetes storageclasses to interact with its product.

I've been using the iscsi storageClass without issue for several months now.

Recently I decided to start using vcluster. I spun up a couple vclusters and everything is working great. I added the clusters to my global configurations (argocd), which installs addons common to all my clusters, including the truenas storageclass.

In both vclusters I'm seeing the same:

$ k get pods
NAME                                               READY   STATUS             RESTARTS         AGE
democratic-csi-iscsi-controller-5c5566758b-cdfhf   1/5     CrashLoopBackOff   39 (4m45s ago)   31m
democratic-csi-iscsi-node-4flpf                    4/4     Running            0                31m
democratic-csi-iscsi-node-4vnvj                    4/4     Running            0                31m
democratic-csi-iscsi-node-d6lbz                    4/4     Running            0                31m
democratic-csi-iscsi-node-qvrvp                    4/4     Running            0                31m

logs show:

[democratic-csi-iscsi-controller-5c5566758b-cdfhf external-snapshotter] I0724 22:23:38.365779       1 connection.go:186] GRPC response: {}
[democratic-csi-iscsi-controller-5c5566758b-cdfhf external-snapshotter] I0724 22:23:38.365852       1 connection.go:187] GRPC error: rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 502 (Bad Gateway); malformed header: missing HTTP content-type
[democratic-csi-iscsi-controller-5c5566758b-cdfhf external-snapshotter] E0724 22:23:38.365872       1 main.go:180] error getting CSI driver name: rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 502 (Bad Gateway); malformed header: missing HTTP content-type

What did you expect to happen?

Expected democratic-csi-iscsi-controller to install to the two vclusters.

How can we reproduce it (as minimally and precisely as possible)?

It may be possible to spin up a minimal configuration using a helm chart:

• name: democratic-csi version: 0.13.1 repository: https://democratic-csi.github.io/charts

values.yaml

democratic-csi:

  csiDriver:
    # should be globally unique for a given cluster
    name: "org.democratic-csi.iscsi"

  # add note here about volume expansion requirements
  storageClasses:
  - name: freenas-iscsi-csi
    defaultClass: true
    reclaimPolicy: Delete
    volumeBindingMode: Immediate
    allowVolumeExpansion: true

Anything else we need to know?

Is there a trick to installing the storageClasses in a vcluster? I mean, I know the storageClass is already there on the root server so we don't really need to install the storageClass again. But, I was thinking I am supposed to install it again?

Host cluster Kubernetes version

```console $ k version -o yaml clientVersion: buildDate: "2022-05-24T12:26:19Z" compiler: gc gitCommit: 3ddd0f45aa91e2f30c70734b175631bec5b5825a gitTreeState: clean gitVersion: v1.24.1 goVersion: go1.18.2 major: "1" minor: "24" platform: linux/amd64 kustomizeVersion: v4.5.4 serverVersion: buildDate: "2022-05-24T12:18:48Z" compiler: gc gitCommit: 3ddd0f45aa91e2f30c70734b175631bec5b5825a gitTreeState: clean gitVersion: v1.24.1 goVersion: go1.18.2 major: "1" minor: "24" platform: linux/amd64 ```

Host cluster Kubernetes distribution

``` kubeadm ```

vcluster version

```console $ vcluster --version vcluster version 0.10.2 ```

Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)

``` k8s ```

OS and Arch

``` OS: centos-8-stream Arch: amd ```

[matskiv]

Hello @lknite I am not sure what's wrong with the democratic-csi-iscsi-controller in this case.

Usually, you would install CSI controller only in the host cluster, but not in the vcluster. And in vcluster you would create only the StorageClass resource, and if it differs from StorageClasses in the host, then you would need to enable storageclasses sync as described in the docs. Alternatively, if you want vcluster to have exactly the same StorageClasses as in the host cluster, you can enable the "legacy-storageclasses" sync as described in the docs.

[lknite]

Thank you. This is what I was looking for, the way things should work. At the moment I still need to use a flexvolume so I think I am stuck with using the legacy-storageclasses. Need to think my setup so I can align more with how things are intended to be used.

Thank you!

[lknite]

I found an updated smb driver so I no longer need to use the flexVolume. Taking another shot at setting this up the right way.

To avoid making any assumptions. If I'm hearing correctly, preferably, I would install my democractic iscsi csi on my host cluster, then setup my vcluster without any storageclass sync. Then on my host cluster do 'k get storageclass -o yaml > sc.yaml', then switch to my vcluster and 'k apply -f sc.yaml'. Yes?

And, if that doesn't work then can try those other options.

[matskiv]

@lknite yes, that should work :). Also, as far as I remember, if you are not syncing StorageClasses from virtual to host then the spec, labels, annotations, etc. of the StorageClasses in vcluster are not important, and are affectively ignored. A StorageClass just needs to exist with the same name as one in the host cluster. Also, if you want to automate this, you can put your sc.yaml into values.yaml for vcluster. Docs - https://www.vcluster.com/docs/operator/init-manifests

[lknite]

Thank you for the tip to add the sc.yaml to the values.yaml, that's what I need to do. Thanks again!