search

newsnowlabs / docker-ingress-routing-daemon

Docker swarm daemon that modifies ingress mesh routing to expose true client IPs to service containers
MIT License
179 stars 34 forks source link

Node IPs are not automatically detected (dird-global) #37

Open megaezz opened 3 months ago

megaezz commented 3 months ago

Hello, friend. I'm trying to use DIRD as docker global service. I have several nodes: 10.0.0.2, 10.0.0.10, 10.0.0.26, 10.0.0.40 I created a service: docker service create --name=dird-global --mode=global --env="DOCKER_NODE_HOSTNAME={{.Node.Hostname}}" --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock newsnowlabs/dird:latest --global-service --install --preexisting But only node with ip 10.0.0.2 got real client ip. Other nodes doesn't. Because service doesn't detect node ips automatically and by default apply rules for nodes 10.0.0.2 - 10.0.0.9 - I see it from logs. So that's why 10.0.0.2 works and other doesn't.

For example, logs of node 10.0.0.26:

2024-06-06.22:04:29.345030|5c8207ded8dc|21710| Docker Ingress Routing Daemon 4.2.0 starting, with args: --install --preexisting
2024-06-06.22:04:29.448612|5c8207ded8dc|21710| Detecting ingress network and node IP:
2024-06-06.22:04:29.453620|5c8207ded8dc|21710| - Ingress subnet: 10.0.0.0/24
2024-06-06.22:04:29.458686|5c8207ded8dc|21710| - This node's IP: 10.0.0.26
2024-06-06.22:04:29.463785|5c8207ded8dc|21710| Cleaning up any stale load-balancer rules ...
2024-06-06.22:04:29.495699|5c8207ded8dc|21710| !!! -------------------------- WARNING ------------------------------------
2024-06-06.22:04:29.500513|5c8207ded8dc|21710| !!! Assuming --ingress-gateway-ips 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.6 10.0.0.7 10.0.0.8 10.0.0.9
2024-06-06.22:04:29.505879|5c8207ded8dc|21710| !!!
2024-06-06.22:04:29.510757|5c8207ded8dc|21710| !!! Please compile a list of the ingress network IPs of each of your nodes
2024-06-06.22:04:29.515838|5c8207ded8dc|21710| !!! that you will be using as a load-balancer.
2024-06-06.22:04:29.520827|5c8207ded8dc|21710| !!!
2024-06-06.22:04:29.525623|5c8207ded8dc|21710| !!! You only have to do this once, or whenever you change your set of
2024-06-06.22:04:29.530684|5c8207ded8dc|21710| !!! load-balancer nodes.
2024-06-06.22:04:29.535341|5c8207ded8dc|21710| !!!
2024-06-06.22:04:29.540332|5c8207ded8dc|21710| !!! Then relaunch using:
2024-06-06.22:04:29.545040|5c8207ded8dc|21710| !!! /opt/docker-ingress-routing-daemon --install --ingress-gateway-ips "<Node Ingress IP List>"
2024-06-06.22:04:29.552642|5c8207ded8dc|21710| !!! ----------------------------------------------------------------------
2024-06-06.22:04:29.565163|5c8207ded8dc|21710| Enumerating load balancers from --ingress-gateway-ips 10.0.0.2 10.0.0.3 10.0.0.4 10.0.0.5 10.0.0.6 10.0.0.7 10.0.0.8 10.0.0.9
2024-06-06.22:04:29.586411|5c8207ded8dc|21710| - Load balancer 10.0.0.2 will have ID 2
2024-06-06.22:04:29.600651|5c8207ded8dc|21710| - Load balancer 10.0.0.3 will have ID 3
2024-06-06.22:04:29.608568|5c8207ded8dc|21710| - Load balancer 10.0.0.4 will have ID 4
2024-06-06.22:04:29.616590|5c8207ded8dc|21710| - Load balancer 10.0.0.5 will have ID 5
2024-06-06.22:04:29.624547|5c8207ded8dc|21710| - Load balancer 10.0.0.6 will have ID 6
2024-06-06.22:04:29.645762|5c8207ded8dc|21710| - Load balancer 10.0.0.7 will have ID 7
2024-06-06.22:04:29.656456|5c8207ded8dc|21710| - Load balancer 10.0.0.8 will have ID 8
2024-06-06.22:04:29.666895|5c8207ded8dc|21710| - Load balancer 10.0.0.9 will have ID 9
2024-06-06.22:04:29.673977|5c8207ded8dc|21710| This node is not a specified load balancer; so skipping installing ingress namespace iptables rules
2024-06-06.22:04:29.679190|5c8207ded8dc|21710| Setting ingress_sbox namespace sysctl variables:
2024-06-06.22:04:29.684294|5c8207ded8dc|21710| - Setting net.ipv4.vs.conn_reuse_mode=0 net.ipv4.vs.expire_nodest_conn=1 net.ipv4.vs.expire_quiescent_template=1
2024-06-06.22:04:29.692749|5c8207ded8dc|21710| Checking for preexisting containers that need routing rules added ...
2024-06-06.22:04:29.890578|5c8207ded8dc|21710| Detected container launch for service 'dird-global', with ID '0562e3fc0ab3' and NID '21527': no ingress network interface found, so skipping.
2024-06-06.22:04:29.896040|5c8207ded8dc|21710| Launching docker event watcher to monitor for container launches (pgroup 21710) ...

How to enable auto detecting of node ips?

struanb commented 3 months ago

Hi @megaezz. Thanks for using DIRD.

DIRD cannot autodetect the node IPs across your clutster, as it can only detect the node IP for the node it is running on.

You seem to know your node IPs though: 10.0.0.2, 10.0.0.10, 10.0.0.26, 10.0.0.40

You could try therefore:

docker service create --name=dird-global --mode=global --env="DOCKER_NODE_HOSTNAME={{.Node.Hostname}}" --mount=type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock newsnowlabs/dird:latest --global-service --install --preexisting --ingress-gateway-ips 10.0.0.2,10.0.0.10,10.0.0.26,10.0.0.40

N.B. Running the newsnowlabs/dird:latest image is not yet officially supported!