search

moby / swarmkit

A toolkit for orchestrating distributed systems at any scale. It includes primitives for node discovery, raft-based consensus, task scheduling and more.
Apache License 2.0
3.38k stars 616 forks source link

Ingress Network limited to 128 ports? #2958

Open ncresswell opened 4 years ago

ncresswell commented 4 years ago

It seems that there is a limitation either with the ingress network, or the docker_gwbridge network, as its seemingly impossible to expose more than 128 ports externally via ingress on a swarm cluster. It does not matter how many nodes comprise the cluster; this issue occurs on a single node swarm cluster or multi-node.

To see this for yourself, on a empty swarm cluster, simply run: docker service create -p 80 nginx 129 times..

128 of them will succeed to provision and expose their ports 30000 - 30128, however on reaching the 129th, the task will set pending forever. If i kill one of the previously deployed services, the one that is pending will instantly deploy.

Note that whilst this limitation exists for the ingress network, it does NOT apply to the standard bridge network. On the same cluster, i was able to deploy 300 containers, each exposing a port via the default bridge network.

Is there system tuning that is required to exceed 128 concurrently exposed ingress ports?

Venomen commented 3 years ago

Hey, any update on this? This is quite critical for bigger swarm instances and annoying.

DavidSche commented 3 years ago

Docker Co need an agreement with Portainer to enhance the docker swarm feature,the docker swarm is for developer ,the kubernetes is for operator

TomaszKorwel commented 3 weeks ago

We are running into this limitation as well. Any chance to get this solved? The linked portainer ticket is marked closed with no resolution.