NetApp / trident

Storage orchestrator for containers
Apache License 2.0
762 stars 222 forks source link

How to select from two backends with the same storage driver? #938

Closed bdf-catalyse closed 1 month ago

bdf-catalyse commented 1 month ago

Describe the solution you'd like We would like a volume to be able to choose betweens different backends with the same storage driver.

AFAIK, if two storage classes have the same backendType parameter, their volumes will end up in the same backends.

We have two ontap-san-economy backends named like iscsi-west and iscsi-east. Is there some trick to define two different storage classes trident-west and trident-east, each storage class using one specific backend?

Describe alternatives you've considered

We could rely on CSI topology, but this would restrict each worker to one of the two backends. In this case, we want that all workers have access to both backends.

Additional context

A similar problem arises when we want to use both iSCSI and NVMe with the same backend type. Today NVMe only supports ontap-san, but as soon as it supports ontap-san-economy, how to define trident-iscsi and trident-nvme storage classes both with backend type ontap-san-economy?

wonderland commented 1 month ago

Multiple ways to approach this, but I'd say the easiest for your use case is the use of storage pools. Trident discovers them automatically, for an Ontap backend they are in the form of "backend-name:aggregate-name". Regular expressions are allowed, so if you only want it to point to a specific backend, but not a specific aggregate, just use ".*" for the aggregate.

Based on your example above, a storage class could look like this:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: trident-west
provisioner: csi.trident.netapp.io
parameters:
  fsType: ext4
  backendType: "ontap-san-economy"
  storagePools: 'iscsi-west:.*'
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
clintonk commented 1 month ago

You can also use labels in the backend definition

labels:
  location: iscsi-east

and label selectors in the storage classes

parameters:
  selector: "location=iscsi-east"
bdf-catalyse commented 1 month ago

Both solutions work. I'll go with labels.