[EKS] [request]: Free IPs in subnet for CP upgrade

[yrotilio]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request Could you change the number of free IPs needed in a subnet to launch a control-plane update to 3, as stated by the documentation ?

Which service(s) is this request for? EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? In order to manage the number of IPs consumed by our EKS architecture, we have limited our control-plane/nodes subnets to 3 /28 CIDR Blocks. We assumed from this architecture that we could manage up to at least 20 nodes without being dependent on node rolling out strategy. Sadly, on cluster with more than 15 nodes, control-plane update fails with the following error message : Error: error updating EKS Cluster (my-cluster) version: InvalidRequestException: Provided subnets subnet-xxxxxxxxxx Free IPs: 4 subnet-xxxxxxxx Free IPs: 4 , need at least 5 IPs in each subnet to be free for this operation What bums me most is that documentation states that "To update the cluster, Amazon EKS requires two to three free IP addresses from the subnets", which would meet our goals, and yet the limit has been set to 5.

Are you currently working around this issue? We have to downscale our cluster until the control-plane update is in progress.

Additional context Ultimately https://github.com/aws/containers-roadmap/issues/170 would be a nice addition for us to manage any subnet size related limitation.

Attachments N/A

[mikestef9]

We'll get those docs updated (https://github.com/awsdocs/amazon-eks-user-guide/pull/531)

In our VPC considerations doc page https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html, we recommend separating subnets passed as part of cluster creation from subnets where you run worker nodes

Make sure that the subnets that you specify during cluster creation have enough available IP addresses for the Amazon EKS created network interfaces. If you're going to deploy a cluster that uses the IPv4 family, we recommend creating small (/28), dedicated subnets for Amazon EKS created network interfaces, and only specifying these subnets as part of cluster creation. Other resources, such as nodes and load balancers, should be launched in separate subnets from the subnets specified during cluster creation.

That way, the size of your cluster does not potentially impact the upgrade process.

[ptwohig]

I'm trying to run all IPv6 and I'm getting this error. Is tehre some caveat with IPv6?