search

kubernetes / ingress-nginx

Ingress-NGINX Controller for Kubernetes
https://kubernetes.github.io/ingress-nginx/
Apache License 2.0
17.26k stars 8.21k forks source link

Question: nginx - multiple replicas or daemonSet? #875

Closed yawboateng closed 7 years ago

yawboateng commented 7 years ago

Am using a deployment with a minimum of 3 replicas and horizontal pod autoscaler enabled with a max of 20 pods and targetCPUUtilizationPercentage 90.

From your experiences is this set up recommended to use with the nginx controller? or should i be using a daemonSet and deploying 1 pod per worker node? or a deployment with a single pod for the entire cluster?

aledbf commented 7 years ago

@yawboateng using a daemonSet is ok if your cluster is "small" (In a cluster with 50 nodes it makes no sense). What you should try to avoid is to locate multiple pods of the ingress controller in the same node.

aledbf commented 7 years ago

Closing. Please reopen if you have more questions

yawboateng commented 7 years ago

thanks @aledbf, will you consider 15 - 20 nodes a small enough for daemonSet?

aledbf commented 7 years ago

@yawboateng I think more than 10 nodes is too much. That said this number depends on the load you need to handle.

jevgenij-alterman commented 5 years ago

@yawboateng using a daemonSet is ok if your cluster is "small" (In a cluster with 50 nodes it makes no sense). What you should try to avoid is to locate multiple pods of the ingress controller in the same node.

can you please explain why?

posquit0 commented 5 years ago

I also wonder why I should avoid is to locate multiple ingress-nginx pods in the same node.

In my case, I isolated k8s node groups for ingress controllers using taint and affinity. And I wonder which is better one of the following designs for production.

Please give me some advice.

aledbf commented 5 years ago

@jevgenij-alterman @posquit0 the reason is simple: you don't need a high number of NGINX instances to handle high volumes of traffic and most importantly, you need to keep in mind that each instance of the ingress controller needs to reach the kubernetes API server. This means if you have lots of replicas, you are putting (unnecessary) pressure. Using a deployment with an anti-affinity rule to avoid multiple replicas in the same node is, in most of the cases, more than enough.

yazbekhe commented 10 months ago

@aledbf i am trying to understand why i should not have more than ONE ingress pod on same node

Pbenteu commented 7 months ago

@yazbekhe Its just a matter of reliability, if you have 2 nginx replicas but they are in the same node if the node fails both replicas will go down and possibly stop traffic