search

NetApp / trident

Storage orchestrator for containers
Apache License 2.0
761 stars 223 forks source link

Multi-tenancy: Quotas and Limits #622

Open mekza opened 3 years ago

mekza commented 3 years ago

Hello,

I can't find any issue or documentation on that specific topic. The context is the following: multiple k8s clusters consuming storage. I would like to know how to deal with multi-tenancy and especially how to set quotas or limits to tenants

Cheers

balaramesh commented 3 years ago

Hello @mekza

Sounds like your question is around best practices/recommendations for multi-tenancy. There's a number of ways you can approach this:

  1. Are multiple k8s clusters consuming data from the same NetApp storage array/subscription? Or is it a case of dedicated SVMs/storage clusters for each k8s cluster?
  2. What is the unit of muti-tenancy here? Are tenants uniquely identified by their namespace? A dedicated k8s cluster?
  3. What kind of quotas/limits are you interested in defining per tenant? Capacity? IOPS?

The most important thing to understand here is that each instance of Trident is unique to the k8s cluster it is installed in. Resource quotas can be established in Kubernetes to limit the number of volumes and total capacity requested per namespace. In addition, CSI Topology is also something you should take a look at. This helps define regions and zones to restrict tenant access/consumption of storage.

It is also recommended you dedicate an SVM per tenant, as that ensures isolation of data across tenants.

mekza commented 3 years ago

Hello @balaramesh

Thanks for your quick reply. The current setup is multiple dedicated clusters k8s consume the same NetApp storage array. The problem I have with Resource quotas is that users can easily override them when they are admin of the given k8s cluster. To sum-up if I understood correctly when each tenant has a SVM, I can set quotas on it (Netapp side), right?

balaramesh commented 3 years ago

@mekza thanks for sharing. As you have identified, establishing quotas through ONTAP requires ONTAP enablement for SVM quotas. This is targeted for a future ONTAP release at this moment.

gnarl commented 3 years ago

@mekza, we are continuing to track this issue. It may be delivered in an ONTAP 9.10 patch release or in ONTAP 9.11. Leaving this issue open for tracking purposes.