newsnowlabs / docker-ingress-routing-daemon

Docker swarm daemon that modifies ingress mesh routing to expose true client IPs to service containers
MIT License
161 stars 31 forks source link

Support ingress subnets larger than /24 #15

Closed struanb closed 2 years ago

struanb commented 2 years ago

DIND can work with a /16 subnet, with caveats: the nodes you are using as load balancers must all have a different final byte in their /16 IP address - but otherwise routing will break.

For example, if your load balancer nodes are x.y.10.2, x.y.11.3 and x.y.12.4, then the TOS byte, which will be set to 2, 3 or 4, can uniquely identify and represent the load balancer in the container firewall and routing rules.

But if one load balancer node was x.y.13.4 then it would clash with x.y.12.4 - both nodes would set the TOS byte to 4, and the containers would set up conflicting and broken firewall and routing rules.

A workaround to make this safe could be as follows: instead of using the final byte of the load balancer ingress IP to determine the TOS byte (as currently), instead generate a TOS byte according to the index of the load balancer IPs in the full list (i.e. the TOS byte should be numbered 1...n where n is the number of load balancer nodes).

These TOS bytes will be unique for each load balancer node and still allow operation with ingress networks larger than /24: /16 or even /8.

struanb commented 2 years ago

A prototype providing full support for ingress networks larger than /24 (e.g. /16) will be pushed to a branch soon.

Testers appreciated.

struanb commented 2 years ago

This has been pushed to the indexed-ids branch here: https://github.com/newsnowlabs/docker-ingress-routing-daemon/blob/indexeds-ids/docker-ingress-routing-daemon

struanb commented 2 years ago

Resolved by https://github.com/newsnowlabs/docker-ingress-routing-daemon/commit/9ca9eab905e7055306c3af3b5803c97d7d5bbbe5.

struanb commented 2 years ago

Released in https://github.com/newsnowlabs/docker-ingress-routing-daemon/releases/tag/v4.1.0