search

uswitch / kiam

Integrate AWS IAM with Kubernetes
Apache License 2.0
1.15k stars 238 forks source link

Compatibility with EKS 1.21 and token service account expiry #515

Open tomsucho opened 2 years ago

tomsucho commented 2 years ago

After our EKS was upgraded to 1.21, we saw annotations like the following appear in api server audit logs in AWS, for service accounts that kiam-server pods are using:

subject: system:serviceaccount::, seconds after warning threshold: 3989 This is due to changes in token expiry in K8s 1.21 as described here: https://docs.aws.amazon.com/eks/latest/userguide/service-accounts.html#identify-pods-using-stale-tokens

It would appear that there is 90d grace period, after which tokens will be rejected. It looks like the kiam server needs to use a later client SDK version, or is there a workaround?

cloudwitch commented 2 years ago

Talked to AWS support about this. They confirmed KIAM 4.2 has high enough Kubernetes Client SDK (v0.20.0) and is good to go from that perspective. This was a worry for us as we're on 3.6.

@tomsucho What KIAM version are you on?

tomsucho commented 2 years ago

@cloudwitch Thanks a lot for checking this! I was actually testing this based on the latest Helm chart which was installing v4.0 I think. And it was still showing up, the annotation. I think it was only reported for kiam-server and not the kiam-agent. I used the Helm chart repo as shown on github:

NAME            CHART VERSION   APP VERSION DESCRIPTION                      
uswitch/kiam    6.1.2           4           Integrate AWS IAM with Kubernetes
h2hoe commented 2 years ago

@cloudwitch @tomsucho Is there a new version of the chart that needs to be released with the updated 4.2 version or will the 4.0 version suffice?

tomsucho commented 2 years ago

@h2hoe in my testing I could see on v4.0 annotations still showing up, so if that is really fixed in 4.2 would be good to get updated chart :)