search

Shopify / kubeaudit

kubeaudit helps you audit your Kubernetes clusters against common security controls
MIT License
1.89k stars 183 forks source link

kubeaudit using unregistered annotations #457

Closed sftim closed 1 year ago

sftim commented 2 years ago
ISSUE TYPE

BUG REPORT

SUMMARY

https://github.com/Shopify/kubeaudit lists some annotations that have meaning to kubeaudit. However, these annotations are not registered and kubeaudit is using them without co-ordinating with the Kubernetes project.

Those annotations:

ENVIRONMENT
STEPS TO REPRODUCE

Visit https://kubernetes.io/docs/reference/labels-annotations-taints/ to see list of registered annotations. Visit https://github.com/Shopify/kubeaudit to see use of these unregistered annotations.

ADDITIONAL INFORMATION

People might assume that these annotations are official, but they actually are not.

FEATURE IDEA

*Proposal:

Either:

ghost commented 2 years ago

Thanks for opening your first issue here! Be sure to follow the issue template!

dani-santos-code commented 2 years ago

Thanks a lot for opening this issue @sftim! Your suggestions sound good. c/c @genevieveluyt

genevieveluyt commented 2 years ago

Thanks for pointing this out!

Kubernetes reserves all labels and annotations in the kubernetes.io namespace

The kubeaudit annotations existed before I started on the project, I had no idea about this! Using a shopify-owned domain sounds to me like the most reasonable solution, but it's going to be quite an annoying breaking change if resources suddenly start triggering kubeaudit so probably we should introduce new annotations and also keep the old ones working for backwards compatibility. Perhaps we can provide a migration path that automatically swaps the annotations.

@sftim Is there any implication for what domain we choose? Based on the registered annotations, they don't appear to actually link to anything. Is using a company-owned domain just a convention?

From my understanding the biggest risk to using unregistered annotations is conflicts, and since we're not aware of anything else using the same annotations as kubeaudit currently this does not have immediate impact?

sftim commented 2 years ago

We really might be able to get registered Kubernetes annotations for these purposes, and document those (even though it's retrospective). The values that the annotation can take might still end up being cluster specific.

I think though it'd be better to pick annotation keys from a key namespace that specifically mentions kubeaudit. If the cost isn't prohibitive, maybe buy a domain name for that?

A common pattern is to deprecate the wrong annotations and log a warning each time (or just the first time) they are spotted, then support them for a while afterwards. A tool to redo those annotations might be handy too.

sftim commented 2 years ago

This is a low impact. Whenever I spot folks using unregistered annotations in the Kubernetes' project's namespace, I ask them to fix it.

sftim commented 2 years ago

Oh, also: label keys are not URIs and there's no expectation that they would be. Public label namespaces (the kubernetes.io in kubernetes.io/example-annotations) are DNS names, and it's polite to use a namespace connected to a domain that exists and doesn't belong to another party.

dani-santos-code commented 1 year ago

This has been addressed in this PR: https://github.com/Shopify/kubeaudit/pull/527

we're going to wait until kubernetes v. 1.28 is released to actually deprecate kubernetes.io in the override labels. we'll only be using kubeaudit.io as suggested.

I will close this issue in favor of this one: https://github.com/Shopify/kubeaudit/issues/533

thank you @sftim! 🙏