search

colinmollenhour / mariadb-galera-swarm

MariaDb Galera Cluster container based on official mariadb image which can auto-bootstrap and recover cluster state.
https://hub.docker.com/r/colinmollenhour/mariadb-galera-swarm
Apache License 2.0
217 stars 103 forks source link

Could not determine NODE_ADDRESS #4

Closed bartoszx closed 7 years ago

bartoszx commented 7 years ago

Helo. Cant find why this doesnt work.

docker service create \ --name mysql-seed \ --replicas 1 \ --network web \ --env MYSQL_ROOT_PASSWORD=xxxx\ --env DISCOVERY_SERVICE=10.155.1.37:2379 \ --env XTRABACKUP_PASSWORD=xxxx \ --env CLUSTER_NAME=bank \ myregistry/abc/bank:mysql seed

Could not determine NODE_ADDRESS: 10.0.0.3 10.0.0.2

The difference between your and mine example is that I use network.

bartoszx commented 7 years ago

I think this is the reason

root@3fc57bb509df:/# ip addr | awk '/inet/ && /eth0/{sub(/\/.*$/,"",$2); print $2}' 10.0.0.5 10.0.0.4

where 10.0.0.4 is VIP and 10.0.0.5 is node ip

colinmollenhour commented 7 years ago

What is the output of just "ip addr"?

bartoszx commented 7 years ago 
root@4ec1c2c7abb2:/# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
193: eth0@if194: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
    link/ether 02:42:0a:00:00:02 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.2/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet 10.0.0.3/32 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:aff:fe00:2/64 scope link
       valid_lft forever preferred_lft forever
195: eth1@if196: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:12:00:05 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.5/16 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe12:5/64 scope link
       valid_lft forever preferred_lft forever
colinmollenhour commented 7 years ago

I quit using Docker Swarm Mode for a while now so not sure if something has changed but I don't remember there being a separate IP reported by "ip addr" for the VIP and the node IP..

Somehow the script needs to be able to determine which IP to consider to be itself and currently there are two IPs output by the awk script which is the problem.. head or tail could be used to "fix" this but I don't even know which one is proper.. Is "web" just a regular "overlay" network or is there something special/different about it?

bartoszx commented 7 years ago

web is docker network create --driver overlay web

the same is for mariadb container

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 5935: eth0@if5936: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default link/ether 02:42:0a:00:00:0d brd ff:ff:ff:ff:ff:ff inet 10.0.0.13/24 scope global eth0 valid_lft forever preferred_lft forever inet 10.0.0.3/32 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:aff:fe00:d/64 scope link valid_lft forever preferred_lft forever 5937: eth1@if5938: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:12:00:05 brd ff:ff:ff:ff:ff:ff inet 172.18.0.5/16 scope global eth1 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe12:5/64 scope link valid_lft forever preferred_lft forever

I think is normal behavior

colinmollenhour commented 7 years ago

Any ideas about how to reliably distinguish between the VIP and the node IP then? I don't currently have a swarm setup to test with.

bartoszx commented 7 years ago

Good question. For fast workaround I took first IP. In every services first IP was node IP. Problem is also with ethX Look at following example. This is another service called nginx

docker service create \ --name nginx \ --replicas 1 \ --publish 80:80 \ --publish 443:443 \ --network web \ registry.edge.do/tvn/xbank:nginx tasks.web

and look that this time overlay web network address is in eth2

root@4b30b036d3b0:/# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 1966: eth0@if1967: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default link/ether 02:42:0a:ff:00:0d brd ff:ff:ff:ff:ff:ff inet 10.255.0.13/16 scope global eth0 valid_lft forever preferred_lft forever inet 10.255.0.9/32 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:aff:feff:d/64 scope link valid_lft forever preferred_lft forever 1968: eth1@if1969: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:12:00:08 brd ff:ff:ff:ff:ff:ff inet 172.18.0.8/16 scope global eth1 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe12:8/64 scope link valid_lft forever preferred_lft forever 1970: eth2@if1971: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default link/ether 02:42:0a:00:00:1e brd ff:ff:ff:ff:ff:ff inet 10.0.0.30/24 scope global eth2 valid_lft forever preferred_lft forever inet 10.0.0.6/32 scope global eth2 valid_lft forever preferred_lft forever inet6 fe80::42:aff:fe00:1e/64 scope link

to be honest I dont know why ingress network is visible here in eth0 I've created all services the same way with web network I have to ask question on docker forums

colinmollenhour commented 7 years ago

You can specify a 'grep -e' pattern like so:

NODE_ADDRESS=^10.0.0.*

Which might help with there being multiple interfaces, but I don't have a solid solution for there being two IPs per interface.. Anything in /etc/hosts that might be helpful?

bartoszx commented 7 years ago

NODE_ADDRESS=^10.0.0.* is good workaround unless you dont specify custom subnet fe docker network create -d overlay --subnet=192.168.0.0/16 web

root@57a2b6d33077:/# cat /etc/hosts 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 10.0.0.2 57a2b6d33077 172.18.0.8 57a2b6d33077

root@4b30b036d3b0:/# cat /etc/hosts 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 10.255.0.13 4b30b036d3b0 172.18.0.8 4b30b036d3b0 10.0.0.30 4b30b036d3b0

colinmollenhour commented 7 years ago

Fixed by PR #8