search

kobolog / gorb

IPVS on steroids – REST API, heartbeats, service discovery and more
GNU Lesser General Public License v3.0
809 stars 83 forks source link

external traffic through virtual port failing to connect to real server #17

Closed BrianAdams closed 8 years ago

BrianAdams commented 8 years ago

I cannot connect to the NAT'd service behind the NLVS box from an outside client. I can however connect to the service using the NLVS IP/Port when I am on the NLVS box directly. Not sure where the config is off, some odd networking side effects in docker 1.9, or if there is a missing dependency for IPVS but it is not working with a SWARM cluster out of the box as a result. Any ideas?

I originally thought the NAT was not working correctly but according to http://www.ultramonkey.org/papers/lvs_tutorial/html/ that tcpdump I was reviewing looks correct.

I have a two host setup with ipvs running on node-1 and the service running on node-2. The node-1 appears to correctly have the routing rule to forward to node-2 setup by gorb-docker-link.

IPVS setting:

TCP 107.170.251.157:9292 wrr -> 104.236.180.219:32768 Masq 100 0 0

From TCP dump I see the inbound connection but the service on the real server does not appear to see the traffic.

15:10:27.928913 IP .58501 > 104.236.180.219.32768: ...

I notice that the outbound traffic is supposed to route back through the NAT director which is typically done by making it a gateway. Anyone has a working swarm setup that still runs NLVS directly on the docker hosts but allows NAT to work? Is it easier to simply go all the way and configure direct routing so to eliminate the need of routing back through the director?

BrianAdams commented 8 years ago

I'll close this for now as it's not a bug but rather a need for some boiler plate patterns for using with typical swarm setups.

kobolog commented 8 years ago

You're right in that in NAT mode the traffic from backends should be routed back to the balancer by setting it as a default gateway for all backends. To avoid this requirement, either IPIP or DR should be used – but they come with their own requirements.