EKS: Planning upgrade to EKS 1.26

[sablumiah]

Go through the release notes of EKS 1.26 and create a plan to upgrade our clusters

Things to consider: Are there new components being added? What changes are being introduced to current components? Are there changes to core infra of the CP required? i.e. Are all our current components compatible with 1.26? Are there changes users need to make? Do we need to expand any of our smoke/integration testing?

https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-kubernetes-version-1-26/

[mikebell]

Deprecated resources:

__________________________________________________________________________________________
>>> Deprecated APIs removed in 1.26 <<<
------------------------------------------------------------------------------------------
KIND                      NAMESPACE   NAME                                           API_VERSION           REPLACE_WITH (SINCE)
HorizontalPodAutoscaler   sm-dev      sm-cloud-platform-canary-application-podinfo   autoscaling/v2beta2   autoscaling/v2 (1.23.0)

Tests: fine on 1.26

Issues:

• When planning after the upgrade aws_eks_cluster platform_version goes from eks.13 to eks.12 - Note AWS EKS starts from 1 with new K8s versions so this is fine just confusing.

• When a new Kubernetes minor version is available in Amazon EKS, such as 1.28, the initial Amazon EKS platform version for that Kubernetes minor version starts at eks.1. However, Amazon EKS releases new platform versions periodically to enable new Kubernetes control plane settings and to provide security fixes.

• Two new IAM policies are added which aren't tracked in terraform

[mikebell]

HorizontalPodAutoscaler   <undefined>                   court-hearing-event-receiver                                      autoscaling/v2beta1   autoscaling/v2 (1.23.0)
HorizontalPodAutoscaler   <undefined>                   hmpps-person-match-score                                          autoscaling/v2beta1   autoscaling/v2 (1.23.0)