Vanilla flavor

[sofixa]

Hey everyone,

There is a (vanilla)[https://github.com/hpe-storage/csi-driver/tree/8874fe968539c6f42c2d7fa6eccd52b584cdf0a8/pkg/flavor/vanilla] (so non-Kubernetes) flavor of this CSI plugin. which seems to be mostly empty boilerplate (including in parts which can totally be implemented without any Kubernetes, like GetChapXFromEnvironment, cf. https://github.com/hpe-storage/csi-driver/blob/8874fe968539c6f42c2d7fa6eccd52b584cdf0a8/pkg/flavor/vanilla/flavor.go#L107-L113).

Is there a way to get this plugin to work on non-Kubernetes environments? Mostly thinking of (Nomad)[https://www.nomadproject.io/]. It successfully runs, but I'm not sure it would actually do anything going off the vanilla/flavor.go.

Cheers, Adrian

[datamattsson]

Theoretically the driver would run on Nomad and other non-Kubernetes environments but the CSI driver would still need access to a Kubernetes cluster to store and manage metadata in CRDs. So, this is a non-starter for most projects.

[sofixa]

@datamattsson since there are flavours handled in the code, shouldn't it be possible to pass this metadata in other ways?

[datamattsson]

Anything is possible, I'm just stating how things sit today.

[sofixa]

As far as I understand things, it's other things around the CSI plugin (like the operator) that translate those CRDs and annotations to native gRPC CSI calls, which the CSI plugin handles. If the CSI plugin is compatible with the CSI spec, it should just work out of the box with any other CSI-compatible orchestrator (as long as there is a way to pass the same metadata that is configured via CRDs in Kubernetes).

In any case, I'm trying this out one stumbling block at a time and will report back. I was hoping there was some doc hidden somewhere - after all, the plugin authors bothered implementing a vanilla flavour in the first place.

[datamattsson]

after all, the plugin authors bothered implementing a vanilla flavour in the first place.

Yes, the plans were grand. One storage framework to rule them all. The CRD I mention, HPENodeInfos, is abstracted in flavor.go through some of the functions. You just have to store that data in your CO somewhere and write a function to store and retrieve it.

Bear in mind that this is just the CSI driver. The CSPs (backends) are notorious for performing layer violations. The 3PAR CSP as an example, accesses the Kubernetes API directly to manage its own CRDs for various metadata resources, so if you're using a 3PAR-based backend, you will have to write a 3PAR CSP that doesn't break the rules as well.

[sofixa]

The CSPs (backends) are notorious for performing layer violations

Seems on par with a CSI node plugin's Helm deployment mounting / in the container to add systemd services that run bash scripts :D

https://github.com/hpe-storage/co-deployments/blob/master/helm/charts/hpe-csi-driver/templates/hpe-csi-node.yaml#L173C1-L196

https://github.com/hpe-storage/csi-driver/blob/8874fe968539c6f42c2d7fa6eccd52b584cdf0a8/cmd/csi-driver/conform/hpe-storage-node.sh

That being said, it's unfortunate. CSI was built on purpose as a vendor-neutral thing, same as CNI.

[datamattsson]

Seems on par with a CSI node plugin's Helm deployment mounting / in the container to add systemd services that run bash scripts :D

Many vendors do this as it was required for certification with Google Anthos a few years back. It can easily be disabled in the Helm chart or using environment variables. Then the user must configure the nodes themselves, we have those steps documented here: https://scod.hpedev.io/csi_driver/operations.html#manual_node_configuration

The flaw in the CSI spec is that there's no interface for configuring the node that require the CSI node driver. Every vendor has their own hacks getting their configurations into place. Surprisingly, the CSI Proxy used for Windows is really what we need for all types of nodes: https://github.com/kubernetes-csi/csi-proxy

Another point in this discussion is that Sidero Labs (the folks behind Talos) were working on a node configuration API from within a CO and making that universally available to other Linux distros is really what the world needs. You can revel in the details as their founder talks about it on the Kubernetes Podcast: https://kubernetespodcast.com/episode/159-talos/

[sofixa]

Agreed, currently configuring nodes is either "do it yourself" or weird tricks like mounting host paths and bashing away. A declarative API for that part would be amazing. I'll check out the podcast you linked, sound good on paper, although Talos' Kubernetes exclusive focus makes me somewhat doubtful they would actively work to support other orchestrators.