search

kubernetes / ingress-nginx

Ingress-NGINX Controller for Kubernetes
https://kubernetes.github.io/ingress-nginx/
Apache License 2.0
17k stars 8.15k forks source link

[feat]: Separate AdmissionWebhooks to its own Host-Networked Pods to Enable Horizontal Scaling in Overlay Networks #10189

Open venkatamutyala opened 1 year ago

venkatamutyala commented 1 year ago

Currently, the admissionWebhooks run in the same pods as the primary NGINX services. This configuration works well in many scenarios, but poses a challenge when using a full overlay network, like Calico, in an EKS environment and there's a need to scale NGINX horizontally.

The control plane nodes in EKS are unable to communicate directly with services on the overlay network. A workaround to this issue is to run the services on the hostNetwork. However, this approach has its limitations. When services run on the hostNetwork, they must allocate a unique port on the node. This effectively restricts us to running only one instance of NGINX per node.

Consider a scenario where I want to scale NGINX to 5 pods across 3 nodes. This becomes impossible due to the port exhaustion on each node, thus limiting the horizontal scaling of the service.

To resolve this issue, I propose that the admissionWebhooks service be decoupled from the main NGINX service and run in its own set of pods. These dedicated pods would run on the hostNetwork, specifically serving the role of validating incoming webhooks. This change would allow the primary NGINX service to continue running on the overlay network, thereby enabling effective horizontal scaling while maintaining necessary communication with the Kubernetes API server for admission control.

k8s-ci-robot commented 1 year ago

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
github-actions[bot] commented 11 months ago

This is stale, but we won't close it automatically, just bare in mind the maintainers may be busy with other tasks and will reach your issue ASAP. If you have any question or request to prioritize this, please reach #ingress-nginx-dev on Kubernetes Slack.