search

hazelcast / hazelcast-kubernetes

Kubernetes Discovery for Hazelcast
Apache License 2.0
174 stars 99 forks source link

Kubernetes service account token with expiry time from Kubernetes version 1.21 #385

Closed z-guohui-qian closed 2 years ago

z-guohui-qian commented 2 years ago

Description

Hello,

We have identified applications running in one or more of your Amazon EKS clusters that are not refreshing service account tokens. Applications making requests to Kubernetes API server with expired tokens will fail. You can resolve the issue by updating your application and its dependencies to use newer versions of Kubernetes client SDK that automatically refreshes the tokens.

What is the problem?

Kubernetes version 1.21 graduated BoundServiceAccountTokenVolume feature to beta and enabled it by default. This feature improves security of service account tokens by requiring a one hour expiry time, over the previous default of no expiration. This means that applications that do not refetch service account tokens periodically will receive an HTTP 401 unauthorized error response on requests to Kubernetes API server with expired tokens. You can learn more about the BoundServiceAccountToken feature in EKS Kubernetes 1.21 release notes.

To enable a smooth migration of applications to the newer time-bound service account tokens, EKS v1.21+ extends the lifetime of service account tokens to 90 days. Applications on EKS v1.21+ clusters that make API server requests with tokens that are older than 90 days will receive an HTTP 401 unauthorized error response.

How can you resolve the issue?

To make the transition to time bound service account tokens easier, Kubernetes has updated the below official versions of client SDKs to automatically refetch tokens before the one hour expiration:

Go v0.15.7 and later Python v12.0.0 and later Java v9.0.0 and later Javascript v0.10.3 and later Ruby master branch Haskell v0.3.0.0 We recommend that you update your application and its dependencies to use one of the above client SDK versions if you are on an older version.

As of April 20th 2022, we have identified the below service accounts attached to pods in one or more of your EKS clusters using stale (older than 1 hour) tokens. Service accounts are listed in the format : |

|kube-system:newrelic-logging |kube-system:newrelic-infrastructure-nrk8s-controlplane

Same issue: https://github.com/newrelic/nri-kubernetes/issues/434

tmurakam commented 2 years ago

Related to https://github.com/hazelcast/hazelcast/issues/21461