search

kubernetes-sigs / aws-ebs-csi-driver

CSI driver for Amazon EBS https://aws.amazon.com/ebs/
Apache License 2.0
997 stars 800 forks source link

Support specifying volume tags via PVC labels or annotations #1876

Open tsjnsn opened 11 months ago

tsjnsn commented 11 months ago

Is your feature request related to a problem?/Why is this needed Applying unique sets of tags based on application requirements currently requires creating a storageclass specific to the application and set of tags. This makes it difficult to maintain tags when there is a lot of variety in keys and values

/feature

Describe the solution you'd like in detail Ideally, you could specify list of tags in the PVC, that way the storage class can remain application-agnostic. One of the following options should work -

Describe alternatives you've considered

Additional context

TomBillietKlarrio commented 10 months ago

also: it would allow tags to be changed. Since you can't update storageclasses (they're immutable)

michealliang123 commented 9 months ago

If we can use PVC lable against sc in dynamic provisioning,This is great for maintaining your own tags independently between each project!

torredil commented 7 months ago

Hi all, thanks for the feature request - we've made a note of this and will evaluate it in the near future.

One key consideration is that there is currently a limitation in Kubernetes where CSI drivers do not have the ability to retrieve tags applied to a PVC from the CreateVolume request passed in by the external provisioner.

We will provide more updates as we make progress on the evaluation and any plans to support this, thanks!

/kind feature

ishaankalra commented 5 months ago

@torredil Till then, cant we have the labels/annotation part at least, considering it cannot be changed once created?

nickvanw commented 5 months ago

If it's useful, we (PlanetScale) worked around this limitation by deploying https://github.com/mtougeron/k8s-pvc-tagger which supports AWS and GCP

ConnorJC3 commented 4 months ago

Hi all, after some discussion, we don't currently plan to support this in the driver itself.

The Kubernetes community intentionally does not supply annotations/labels to CSI drivers: https://github.com/kubernetes-csi/external-provisioner/issues/86. Because of this, the only solution would involve a work-around that directly bypasses this design and introduces race condition and correctness concerns.

We do support dynamic tag values based on the PVC name and namespace, and the PV name: https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/tagging.md#storageclass-tagging. We have seen these features utilized in usecases such as cost tagging and tagging volumes for DLM policies.

We are working on the ability to modify the tags of existing volumes using Kubernetes's VolumeAttributesClass feature. We expect to release this capability sometime in the next few EBS CSI releases. (Note, however, that because VACs are an alpha feature they are not yet supported by EKS. We are engaging with the upstream community to push VAC to beta/GA in future Kubernetes releases.)

k8s-triage-robot commented 1 month ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

TomBillietKlarrio commented 1 month ago

The VolumeAttributesClass would be a really nice solution. Since it's in Beta now in 1.31, any roadmap on when the CSI driver would support it? For what it's worth, we currently use https://github.com/mtougeron/k8s-pvc-tagger as a "workaround"

ConnorJC3 commented 1 month ago

@TomBillietKlarrio modifying tags via VolumeAttributesClass is available in the EBS CSI Driver today, see https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/tagging.md#adding-modifying-and-deleting-tags-of-existing-volumes

Using dynamic substitution in those tags as can be done in the StorageClass (with the PVC and PV name/namespace) is not yet supported due to us waiting on a required upstream change, but should be available in a future release of the EBS CSI Driver.

k8s-triage-robot commented 3 weeks ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

AndrewSirenko commented 3 weeks ago

/remove-lifecycle rotten

Can modify tags via VAC now, see Volume modification via VolumeAttributesClass example.

We're waiting to close issue until #2093 is merged (interpolated tags), which will happen once the next kubernetes-csi/external-resizer is released after K8s v1.32.

morremeyer commented 3 days ago

Hey everyone, just to clarify that I understood everything in here correctly:

If I get all this correctly, what is the recommended way to do configure per-volume tags that can not be inferred from PVC name, namespace or PV name?

We do have a requirement to set tags like e.g. owner for our internal cost attribution, which is not possible with any of the above since the owner is not inferred from the namespace necessarily in our case.

Note: As far as I'm aware, it would technically be possible to create one VAC per volume, but as far as I understand the design, that is not the intended use.