search

nforgeio / neonKUBE

Public NeonKUBE Kubernetes distribution related projects
https://neonkube.io
Apache License 2.0
76 stars 13 forks source link

Remove dependency on LegacyServiceAccountTokenNoAutoGeneration=false #1870

Closed jefflill closed 7 months ago

jefflill commented 7 months ago

Starting with Kubernetes 1.25, the LegacyServiceAccountTokenNoAutoGeneration=true feature gate defaults to TRUE which breaks cluster setup and probably a lot of user Helm charts etc. by disabling automatic creation of service account tokens. This is a security enhancement that provides for service token expiration and rotation but requires that service tokens be explicitly created via the TokenRequest API.

Kubernetes already provisions JWTs to workloads. This functionality is on by default and thus widely deployed. The current workload JWT system has serious issues:

  1. Security: JWTs are not audience bound. Any recipient of a JWT can masquerade as the presenter to anyone else.
  2. Security: The current model of storing the service account token in a Secret and delivering it to nodes results in a broad attack surface for the Kubernetes control plane when powerful components are run - giving a service account a permission means that any component that can see that service account's secrets is at least as powerful as the component.
  3. Security: JWTs are not time bound. A JWT compromised via 1 or 2, is valid for as long as the service account exists. This may be mitigated with service account signing key rotation but is not supported by client-go and not automated by the control plane and thus is not widely deployed.
  4. Scalability: JWTs require a Kubernetes secret per service account.

https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/1205-bound-service-account-tokens https://www.linkedin.com/pulse/service-account-token-changes-kubernetes-version-124-shafeeque-aslam https://programmingwithwolfgang.com/use-the-tokenrequest-api-to-create-tokens-in-kubernetes/

I've added LegacyServiceAccountTokenNoAutoGeneration=false as one of the required feature gates. We should look into removing this in the future.

jefflill commented 7 months ago

I was hoping I could set the LegacyServiceAccountTokenNoAutoGeneration=false feature gate but it turns out that Kubernetes doesn't allow disabling GA features. I guess this makes sense because otherwise, the Kubernetes team would need to keep old feature code like this around.

I've updated our feature gate checks to handle this but unfortunately, this means we'll need to deal with TokenRequests immediately.

jefflill commented 7 months ago

CLOSING: After further investigation, this wasn't actually the problem. I've gotten past this with the cilium port.