Edge MAC is different and causes network failure

[russ1217]

Hi, there,

I have an odd situation: using [edge -d -a -c -k -l] command to generate a TAP device with a random MAC address, e.g. BE:CB:61:90:D3:20. However, after the command finishes successfully, ifconfig shows the TAP device's actual MAC is a different one, e.g. 8a:83:34:66:45:d7. The TAP device cannot function. e.g. cannot be pinged. Even when I use -m option with edge to set MAC manually, it still happens. BTW, My supernode uses -M option.

If I use [#ifconfig TAP_NAME hw ether BE:CB:61:90:D3:20] command to forcefully set its MAC address to what edge command reports, the TAP device works. Is there anyway to fix this?

[Logan007]

Reminds me of #659.

[GreatMichaelLee]

same issue to me

[russ1217]

You are very likely to be right. I forgot to mention that mine also happens on VPS.

[russ1217]

i think i found a solution to this at https://bugzilla.suse.com/show_bug.cgi?id=1136600

edit [/usr/lib/systemd/network/99-default.link] change [MACAddressPolicy=persistent to MACAddressPolicy=none] run [sudo systemctl restart systemd-networkd]

[hamishcoleman]

Looking a little closer here, the systemd/udev defaults used for tap devices was changed in v242 (about April 2019) and became MACAddressPolicy=persistent

The above workaround is a pretty big hammer as it turns this feature off for all interfaces - not just the N2N interface, but should work until a better implementation is written.

As suggested in #659, it is probably best to have the edge listen for hwaddr changes and refresh its internal idea of MAC address on detecting a change.

It looks to me that the root cause is that the edge SIOCSIFHWADDR ioctl is racing with the udev MACAddressPolicy. As soon as the hwaddr is set, the udev/systemd code detects that and will leave the new address untouched, but there (by necessity) is a small window between creating the tap device and setting the hwaddr - during this window the udev can be notified that a new interface has appeared and check the addr_assign_type, then get interrupted by the edge daemon before then finishing its work and appling the mac address policy (This is a classic TOCTOU race)

Since there is no locking available for this race, having a dynamic MAC change watcher in the edge daemon would be a good answer - it could also allow some more advanced networking options and the changes needed to support changing the n2n mac address would help some of the future options for replacing tuntap devices (Supporting newer Apple kernel virtual interfaces in particular would probably be easier with this)

[russ1217]

Thanks! very well explained. if the "big hammer" is not used, probably for now the best option would be using [edge -m] to set MAC manually first, and then using [ifconfig] or [ip] command to manually set tuntap device of n2n to the pre-set MAC. At least I found this works.

[hamishcoleman]

That will work, but I personally don't like needing to specify a MAC address in two different files.

If you wanted a slightly smaller hammer, you could create /etc/systemd/network/95-tun.link with

[Match]
Driver=tun

[Link]
MACAddressPolicy=none

As this will apply that policy fix to just the tun/tap devices (If you have a fixed interface name for your n2n device, you can limit this even further by using something like Name=n2n0 in the match section instead of the Driver)

This change should be able to be activated with sudo udevadm control --reload