Multidomain-Feature: MAC-Collissions & Flapping

margau commented 5 years ago

Hello, we (Freifunk FFM) are running mesh-announce with the multidomain-Feature and Yanic as Backend. Unfortunately, we are experiencing MAC-Collisions and "flapping" Gateway-States.

Yanic Log: level="warn" msg="override nodeID from aaff16[] to aaff15[] on MAC address aa:ff:17:[]" caller="nodes.go:206 github.com/FreifunkBremen/yanic/runtime.(*Nodes).readIfaces"

On our Map (map.ffm.freifunk.net) you can see the "flapping" Gateways: They are appearing and disappearing more or less randomly.

respondd.py is running with the following args: ARGS=-d /opt/mesh-announce/providers -b dom13-bat -i dom13-br -i dom13-tp -i dom13-vpn-1312 -b dom4-bat -i dom4-br -i dom4-tp -i dom4-vpn-1312 [...]

Where could the problem be? Thanks margau

genofire commented 5 years ago

you have two gateways with the node_id: aaff16 and aaff15

and on both the interface have the same MAC Address aa:ff:17:xx:xx:xx

you have to change the mac address on on of your gateway -> every MAC should be unique in the whole network

this is not a yanic or mesh-announce problem - it is a problem in your network configuration

margau commented 5 years ago

Hi, no, that aren't different gateways. aaff15[] and aaff16[] are part of two domains running on the same gateway with a single mesh-announce instance as stated above. There are no duplicate MAC's.

Best regards margau

genofire commented 5 years ago

Running mesh-announce multiple time - so that interfaces are duplicated listed

margau commented 5 years ago

No, there is exactly one mesh-announce instance running - as described above - with the multidomain-feature from @TobleMiner.

genofire commented 5 years ago

ip link ?

margau commented 5 years ago

112: dom13-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:13:00:02:04 brd ff:ff:ff:ff:ff:ff

113: dom13-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom13-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:13:00:02:02 brd ff:ff:ff:ff:ff:ff

114: dom13-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom13-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:13:00:02:03 brd ff:ff:ff:ff:ff:ff

115: dom13-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom13-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:13:00:02:08 brd ff:ff:ff:ff:ff:ff

116: dom4-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:04:00:02:04 brd ff:ff:ff:ff:ff:ff

117: dom4-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom4-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:04:00:02:02 brd ff:ff:ff:ff:ff:ff

118: dom4-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom4-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:04:00:02:03 brd ff:ff:ff:ff:ff:ff

119: dom4-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom4-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:04:00:02:08 brd ff:ff:ff:ff:ff:ff

120: dom15-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:15:00:02:04 brd ff:ff:ff:ff:ff:ff

121: dom15-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom15-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:15:00:02:02 brd ff:ff:ff:ff:ff:ff

122: dom15-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom15-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:15:00:02:03 brd ff:ff:ff:ff:ff:ff

123: dom15-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom15-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:15:00:02:08 brd ff:ff:ff:ff:ff:ff

124: dom9-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:09:00:02:04 brd ff:ff:ff:ff:ff:ff

125: dom9-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom9-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:09:00:02:02 brd ff:ff:ff:ff:ff:ff

126: dom9-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom9-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:09:00:02:03 brd ff:ff:ff:ff:ff:ff

127: dom9-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom9-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:09:00:02:08 brd ff:ff:ff:ff:ff:ff

128: dom16-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:16:00:02:04 brd ff:ff:ff:ff:ff:ff

129: dom16-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom16-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:16:00:02:02 brd ff:ff:ff:ff:ff:ff

130: dom16-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom16-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:16:00:02:03 brd ff:ff:ff:ff:ff:ff

131: dom16-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom16-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:16:00:02:08 brd ff:ff:ff:ff:ff:ff

132: dom2-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:02:00:02:04 brd ff:ff:ff:ff:ff:ff

133: dom2-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom2-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:02:00:02:02 brd ff:ff:ff:ff:ff:ff

134: dom2-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom2-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:02:00:02:03 brd ff:ff:ff:ff:ff:ff

135: dom2-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom2-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:02:00:02:08 brd ff:ff:ff:ff:ff:ff

136: dom6-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:06:00:02:04 brd ff:ff:ff:ff:ff:ff

137: dom6-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom6-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:06:00:02:02 brd ff:ff:ff:ff:ff:ff

138: dom6-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom6-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:06:00:02:03 brd ff:ff:ff:ff:ff:ff

139: dom6-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom6-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:06:00:02:08 brd ff:ff:ff:ff:ff:ff

140: dom1-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:01:00:02:04 brd ff:ff:ff:ff:ff:ff

141: dom1-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom1-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:01:00:02:02 brd ff:ff:ff:ff:ff:ff

142: dom1-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom1-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:01:00:02:03 brd ff:ff:ff:ff:ff:ff

143: dom1-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom1-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:01:00:02:08 brd ff:ff:ff:ff:ff:ff

144: dom14-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:14:00:02:04 brd ff:ff:ff:ff:ff:ff

145: dom14-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom14-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:14:00:02:02 brd ff:ff:ff:ff:ff:ff

146: dom14-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom14-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:14:00:02:03 brd ff:ff:ff:ff:ff:ff

147: dom14-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom14-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:14:00:02:08 brd ff:ff:ff:ff:ff:ff

148: dom7-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:07:00:02:04 brd ff:ff:ff:ff:ff:ff

149: dom7-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom7-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:07:00:02:02 brd ff:ff:ff:ff:ff:ff

150: dom7-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom7-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:07:00:02:03 brd ff:ff:ff:ff:ff:ff

151: dom7-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom7-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:07:00:02:08 brd ff:ff:ff:ff:ff:ff

152: dom17-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:17:00:02:04 brd ff:ff:ff:ff:ff:ff

153: dom17-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom17-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:17:00:02:02 brd ff:ff:ff:ff:ff:ff

154: dom17-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom17-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:17:00:02:03 brd ff:ff:ff:ff:ff:ff

155: dom17-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom17-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:17:00:02:08 brd ff:ff:ff:ff:ff:ff

156: dom3-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:03:00:02:04 brd ff:ff:ff:ff:ff:ff

157: dom3-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom3-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:03:00:02:02 brd ff:ff:ff:ff:ff:ff

158: dom3-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom3-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:03:00:02:03 brd ff:ff:ff:ff:ff:ff

159: dom3-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom3-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:03:00:02:08 brd ff:ff:ff:ff:ff:ff

160: dom11-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:11:00:02:04 brd ff:ff:ff:ff:ff:ff

161: dom11-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom11-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:11:00:02:02 brd ff:ff:ff:ff:ff:ff

162: dom11-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom11-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:11:00:02:03 brd ff:ff:ff:ff:ff:ff

163: dom11-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom11-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:11:00:02:08 brd ff:ff:ff:ff:ff:ff

164: dom5-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:05:00:02:04 brd ff:ff:ff:ff:ff:ff

165: dom5-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom5-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:05:00:02:02 brd ff:ff:ff:ff:ff:ff

166: dom5-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom5-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:05:00:02:03 brd ff:ff:ff:ff:ff:ff

167: dom5-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom5-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:05:00:02:08 brd ff:ff:ff:ff:ff:ff

168: dom8-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:08:00:02:04 brd ff:ff:ff:ff:ff:ff

169: dom8-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom8-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:08:00:02:02 brd ff:ff:ff:ff:ff:ff

170: dom8-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom8-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:08:00:02:03 brd ff:ff:ff:ff:ff:ff

171: dom8-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom8-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:08:00:02:08 brd ff:ff:ff:ff:ff:ff

172: dom12-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:12:00:02:04 brd ff:ff:ff:ff:ff:ff

173: dom12-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom12-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:12:00:02:02 brd ff:ff:ff:ff:ff:ff

174: dom12-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom12-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:12:00:02:03 brd ff:ff:ff:ff:ff:ff

175: dom12-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom12-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:12:00:02:08 brd ff:ff:ff:ff:ff:ff

176: dom0-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:00:00:02:04 brd ff:ff:ff:ff:ff:ff

177: dom0-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom0-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:00:00:02:02 brd ff:ff:ff:ff:ff:ff

178: dom0-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom0-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:00:00:02:03 brd ff:ff:ff:ff:ff:ff

179: dom0-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom0-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:00:00:02:08 brd ff:ff:ff:ff:ff:ff

180: dom10-br: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether aa:ff:10:00:02:04 brd ff:ff:ff:ff:ff:ff

181: dom10-bat: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master dom10-br state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:10:00:02:02 brd ff:ff:ff:ff:ff:ff

182: dom10-vpn-1312: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc pfifo_fast master dom10-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fa:ff:10:00:02:03 brd ff:ff:ff:ff:ff:ff

183: dom10-tp: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1312 qdisc noqueue master dom10-bat state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether aa:ff:10:00:02:08 brd ff:ff:ff:ff:ff:ff

Suuush commented 4 years ago

Same issue in Hannover. Is there any fix, workaround or wip?

Regards Sush

TobleMiner commented 4 years ago

@Suuush Unfortunately I do not have a multidomain setup to verify what is happening myself. I can see it on your map though. I'll see what I can do.

margau commented 4 years ago

Our "workaround" is a single mesh-announce-instance per domain, which solves the MAC-Collision-error. Our example for domain 2: python3 /opt/mesh-announce/respondd.py -d /opt/mesh-announce/providers -i dom2-br -i dom2-tp -b dom2-bat

Unfortunately, if we deploy this for all our 18 domains (18 times respondd.py), we have a memleak inside the UDP Receive Buffer, which stops all UDP-Packets from being received after a While (including fastd) and renders the gateway useless.

It seems to work with only one domain mesh-announce actively running, which is enough for our current use case at the moment. The memleak could be related to https://bugs.python.org/issue37193 ?

Suuush commented 4 years ago

@Suuush Unfortunately I do not have a multidomain setup to verify what is happening myself. I can see it on your map though. I'll see what I can do.

Thanks! If you need more information...

AiyionPrime commented 4 years ago

Happy new year everyone; I can confirm, the bug is still alive and healthy.

lemoer commented 4 years ago

@Suuush , I tried to debug the problem and came to she same conclusion...

@TobleMiner Maybe this helps you in debugging. It seems like the information is saved in the wrong node instance, because the node_id and mac are not matching.

lemoer@orange ~> curl -s "https://harvester.ffh.zone/meshviewer.json" | jq '.nodes | map(select(.is_gateway and .hostname == "sn01" )) | .[] | " \(.node_id) \(.mac)" ' | tr -d '"'
 88e640ba1020 88:e6:40:ba:10:19
 88e640ba1010 88:e6:40:ba:10:23
 88e640ba1019 88:e6:40:ba:10:19
 88e640ba1021 88:e6:40:ba:10:19
 88e640ba1013 88:e6:40:ba:10:11
 88e640ba1011 88:e6:40:ba:10:11
 88e640ba1023 88:e6:40:ba:10:23
 88e640ba1016 88:e6:40:ba:10:19
 88e640ba1002 88:e6:40:ba:10:02
 88e640ba1015 88:e6:40:ba:10:15
 88e640ba1012 88:e6:40:ba:10:20
 88e640ba1014 88:e6:40:ba:10:14
 88e640ba1022 88:e6:40:ba:10:22
 88e640ba1018 88:e6:40:ba:10:18
 88e640ba1017 88:e6:40:ba:10:22
 88e640ba1099 88:e6:40:ba:10:99

lemoer@orange ~> curl -s 'https://harvester.ffh.zone/meshviewer.json' | jq '.nodes | map(select(.is_gateway and .hostname == "sn01" )) | map(.mac)' | tr -d ' ' | tr -d ',' | grep -v '\[' | grep -v '\]' | sort | uniq -c
      1 "88:e6:40:ba:10:02"
      1 "88:e6:40:ba:10:10"
      1 "88:e6:40:ba:10:11"
      1 "88:e6:40:ba:10:14"
      1 "88:e6:40:ba:10:17"
      1 "88:e6:40:ba:10:18"
      3 "88:e6:40:ba:10:19"
      1 "88:e6:40:ba:10:20"
      2 "88:e6:40:ba:10:21"
      2 "88:e6:40:ba:10:22"
      1 "88:e6:40:ba:10:23"
      1 "88:e6:40:ba:10:99"

lemoer@orange ~> curl -s 'https://harvester.ffh.zone/meshviewer.json' | jq '.nodes | map(select(.is_gateway and .hostname == "sn01" )) | map(.node_id)' | tr -d ' ' | tr -d ',' | grep -v '\[' | grep -v '\]' | sort | uniq -c
      1 "88e640ba1002"
      1 "88e640ba1010"
      1 "88e640ba1011"
      1 "88e640ba1012"
      1 "88e640ba1013"
      1 "88e640ba1014"
      1 "88e640ba1015"
      1 "88e640ba1016"
      1 "88e640ba1017"
      1 "88e640ba1018"
      1 "88e640ba1019"
      1 "88e640ba1020"
      1 "88e640ba1021"
      1 "88e640ba1022"
      1 "88e640ba1023"
      1 "88e640ba1099"

lemoer commented 4 years ago

I can also verify that this is not a bug of yanic, but mesh-announce:

root@perf-test:~# gluon-neighbour-info -d fe80::8ae6:40ff:feba:1099 -r nodeinfo -p 1001 -i br-client
{
  "node_id": "88e640ba1016",
  "hostname": "sn01",
  "software": {
    "batman-adv": {
      "version": "2018.0-6-g49bb9f75"
    },
    "firmware": {
      "release": "9.11",
      "base": "Debian"
    },
    "fastd": {
      "version": "v18",
      "enabled": true
    }
  },
  "network": {
    "mac": "88:e6:40:ba:10:18",
    "addresses": [
      "fe80::8ae6:40ff:feba:1018",
      "fdca:ffee:8:18::1001",
      "2a02:790:ff:118::1001"
    ],
    "mesh": {
      "bat18": {
        "interfaces": {
          "tunnel": [
            "88:e6:40:20:10:18",
            "4e:9f:86:ac:ab:c4"
          ]
        }
      }
    }
  },
  "hardware": {
    "nproc": 12
  },
  "vpn": true
}

There are three incoherent informations:

The query is from domain99 (aka bat99).
.network.mac is from domain18 (see last octet)
.node_id is from domain16 (see last octet)

lemoer commented 4 years ago

It seems to be an issue of the cache. The following patch seems to fix it:

diff --git a/providers/__init__.py b/providers/__init__.py
index 92d8ced..0947b26 100644
--- a/providers/__init__.py
+++ b/providers/__init__.py
@@ -11,7 +11,7 @@ class DataSource():
     def cache_ttl(self):
         ''' Default cache ttl, set to 0 to disable cache
         '''
-        return 30
+        return 0

     def required_args(self):
         ''' Returns a list of required argument names

AiyionPrime commented 4 years ago

At least in our community the errors exist on all four active supernodes, but in different intensities. The wrong mac adresses that got delivered by them were too often the same to be random.

E.g. on one supernode several interfaces were responded with the same wrong macaddress. Possible that (part of) the caching lacks the ability to support multiple domains, yet.

Tomorrow morning I'll report back how well @lemoers hotfix works out, but for now, it looks great.

If it stays that way, I'd like to propose the possibility of dropping it alltogether from the code.

lemoer commented 4 years ago

I tried to create another patch without disabling the cache. Here's my proposal:

diff --git a/providers/__init__.py b/providers/__init__.py
index c7b6bc1..582287d 100644
--- a/providers/__init__.py
+++ b/providers/__init__.py
@@ -97,16 +97,17 @@ class Source():

     def call(self, env):
         cache = SourceCache.getinstance()
-        cache_result = cache.get(self)
-        if cache_result:
-            return cache_result

         args = []
         for argname in self.source.required_args():
             args.append(env[argname])

+        cache_result = cache.get(frozenset([self] + args))
+        if cache_result:
+            return cache_result
+
         result = self.source.call(*args)
-        cache.put(self, result, self.source.cache_ttl())
+        cache.put(frozenset([self] + args), result, self.source.cache_ttl())

         return result

AiyionPrime commented 4 years ago

I've now reviewed the logs of the last night. @lemoer 's patch results in no more collisons and as a result of that no more overwrites and therefore no more offline nodes. I'll start a pull request with his code.

ffnord / mesh-announce

Multidomain-Feature: MAC-Collissions & Flapping #49