search

aws / aws-eks-best-practices

A best practices guide for day 2 operations, including operational excellence, security, reliability, performance efficiency, and cost optimization.
https://aws.github.io/aws-eks-best-practices/
Other
1.99k stars 483 forks source link

Add recommendations for kube-proxy in IPVS mode #324

Open sheetaljoshi opened 1 year ago

sheetaljoshi commented 1 year ago

Is your idea request related to a problem that you've solved? Please describe. When kube-proxy is running in iptables mode, it creates iptable rules for every Service in the cluster on every worker node. Each Service can have multiple endpoints (pods), so if there are 2000 services and each service has 10 pods, that means there are 20,000 endpoints that kube-proxy needs to create iptable rules for on each worker node.

When there are a large number of iptable rules, the kernel can become busy trying to match packets against those rules. This can lead to performance and latency issues, especially if the worker nodes are running on older or less powerful hardware.

Customers are configuring kube-proxy in IPVS mode to solve the scale issue encountered when using kube-proxy IPVS mode.

Describe the best practice We will add a section for IPVS under networking section and list the recommendation for using IPVS settings and any core-dns settings that need to be considered when enabling kube-proxy in IPVS mode.

Describe alternatives you've considered None

Additional context None

jicowan commented 1 year ago

I believe this is a good idea @sheetaljoshi. I would mention how endpoints slices attempts to solve this problem and why it is not sufficient. I would also include a section on proxy-less CNIs like Cilium and how they can used to address this issue.