search

opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.39k stars 757 forks source link

[17.1] setting a mac on vlan interface over a lagg results in all interfaces getting the mac. #1705

Closed sjorge closed 7 years ago

sjorge commented 7 years ago

Each vlan interface was given the mac of the device it replaced. 00:22:06:ff:10:01 -> lagg0_vlan10 00:22:06:ff:20:01 -> lagg0_vlan20 00:22:06:ff:30:01 -> lagg0_vlan40 00:22:06:ff:40:01 -> lagg0_vlan40

They all end up with the mac, which one it gets seems random :/

root@exosphere:/conf # ifconfig
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
    ether 00:22:06:ff:05:01
    inet6 fe80::215:17ff:fec7:9ae8%em0 prefixlen 64 scopeid 0x1
    inet 192.168.0.163 netmask 0xffffff00 broadcast 192.168.0.255
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6007ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:22:06:ff:30:01
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6007ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:22:06:ff:30:01
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
igb2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6007ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:22:06:ff:30:01
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
igb3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6007ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:22:06:ff:30:01
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
enc0: flags=0<> metric 0 mtu 1536
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: enc
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
    inet 127.0.0.1 netmask 0xff000000
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: lo
pflog0: flags=100<PROMISC> metric 0 mtu 33160
    groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
    groups: pfsync
    syncpeer: 0.0.0.0 maxupd: 128 defer: off
ovpns1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    inet6 fe80::222:6ff:feff:501%ovpns1 prefixlen 64 scopeid 0xa
    inet6 2001:470:7ee7:70:1000::1 prefixlen 68
    inet 10.23.70.1 --> 10.23.70.2  netmask 0xffffffc0
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: tun openvpn
    Opened by PID 10719
ovpns2: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: tun openvpn
ovpns3: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    inet6 fe80::222:6ff:feff:501%ovpns3 prefixlen 64 scopeid 0xc
    inet6 2001:470:7ee7:70:3000::1 prefixlen 68
    inet 10.23.70.129 --> 10.23.70.130  netmask 0xffffffc0
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: tun openvpn
    Opened by PID 51407
ovpns4: flags=8010<POINTOPOINT,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: tun openvpn
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=6007ab<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:22:06:ff:30:01
    inet6 fe80::ec4:7aff:fe32:2784%lagg0 prefixlen 64 scopeid 0xe
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    groups: lagg
    laggproto lacp lagghash l2,l3,l4
    laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    laggport: igb2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
    laggport: igb3 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
lagg0_vlan10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1480
    ether 00:22:06:ff:30:01
    inet6 fe80::ec4:7aff:fe32:2784%lagg0_vlan10 prefixlen 64 scopeid 0xf
    inet6 2001:470:7ee7:10::1 prefixlen 64
    inet 10.23.10.1 netmask 0xffffff00 broadcast 10.23.10.255
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    vlan: 10 vlanpcp: 0 parent interface: lagg0
    groups: vlan
lagg0_vlan20: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1480
    ether 00:22:06:ff:30:01
    inet6 fe80::ec4:7aff:fe32:2784%lagg0_vlan20 prefixlen 64 scopeid 0x10
    inet6 2001:470:7ee7:20::1 prefixlen 64
    inet 10.23.20.1 netmask 0xffffff00 broadcast 10.23.20.255
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    vlan: 20 vlanpcp: 2 parent interface: lagg0
    groups: vlan
lagg0_vlan30: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1480
    ether 00:22:06:ff:30:01
    inet6 fe80::ec4:7aff:fe32:2784%lagg0_vlan30 prefixlen 64 scopeid 0x11
    inet6 2001:470:7ee7:30::1 prefixlen 64
    inet 10.23.30.1 netmask 0xffffff00 broadcast 10.23.30.255
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    vlan: 30 vlanpcp: 0 parent interface: lagg0
    groups: vlan
lagg0_vlan40: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1480
    ether 00:22:06:ff:30:01
    inet6 fe80::ec4:7aff:fe32:2784%lagg0_vlan40 prefixlen 64 scopeid 0x12
    inet6 2001:470:7ee7:40::1 prefixlen 64
    inet 10.23.40.1 netmask 0xffffff00 broadcast 10.23.40.255
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    vlan: 40 vlanpcp: 7 parent interface: lagg0
    groups: vlan
gif0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1280
    options=80000<LINKSTATE>
    tunnel inet 192.168.0.163 --> 216.66.84.46
    inet6 2001:470:1f14:d95::2 --> 2001:470:1f14:d95::1  prefixlen 128
    inet6 fe80::222:6ff:feff:501%gif0 prefixlen 64 scopeid 0x13
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: gif
AdSchellevis commented 7 years ago

@sjorge if I'm not mistaken it should get the mac address of the first lagg member, do you have issues with your setup?

sjorge commented 7 years ago

Yes, none of the IPv6 stuff is working after this.

The setup is router-on-a-stick like. 4x 1GBit lagg0 with multiple tagged vlans over it. With each vif having a spoofed mac, I just rebooted with macspoofing disabled and still no IPv6.

OpenVPN, etc... that bind on those are all down.

I would expect all physical interfaces that are part of the lag to have the mac of the first device. But the vif's should have there own mac I think, not 100% on this since there may be different implementations out there.

AdSchellevis commented 7 years ago

I would start with a simple lagg setup and test step by step, the man pages aren't very clear about the behaviour, but it's quite unlikely lagg interfaces are completely broken. (it could be an issue with the other end as well)

sjorge commented 7 years ago

I'm reverting the stuff now because it is getting late, I will start small on sunday when I should have some time again.

sjorge commented 7 years ago

Quick question, assuming the lagg0 is fine (I see FreeBSD also has the same behavior)... Did something change with OpenVPN? My 2 IPv6 servers now contain the line: One UDP6 and one TCP6

local 10.23.30.1

Which seems to be the reason they are failing to start... I did do a clean install on this box and imported my config, on the old box the line was missing. After a update of the service the line gets added and it is broken again.

Maybe something that happened with the OpenVPN to plugin framework conversion? If so I will close this and file a seperate issue for the OpenVPN bits. I have not looked as to why ntpd and dhcpv6 are failing though.

AdSchellevis commented 7 years ago

The location of the files changed to make them pluggable, but the logic should be the same, the logic to determine "local" is located here https://github.com/opnsense/core/blob/master/src/etc/inc/plugins.inc.d/openvpn.inc#L543-L551.

sjorge commented 7 years ago

I just compared my exported config from the new one to the old one.

The OpenVPN bits are the same and the interface it is listening on (opt3) is the same too. Same IPv4 and IPv6, but one is a vnic in a VM the other one is a vlan over a lagg. I will add some prints around that location you mentioned to see if I can see a difference on Sunday.

Otherwise, even mac (although not sure that works in combination with an lagg) are the same.

fichtner commented 7 years ago

Try

# opnsense-revert -r 17.1.9 opnsense

It's unlikely the plugin rework, maybe a side effect of the new openvpn 2.4

fichtner commented 7 years ago

Errr, I meant 17.1.8

sjorge commented 7 years ago

Well that was stupid, It looks like the download image is not the latest version and since. A upgrade fixed it. Now i just need to get SoL going and I'm all set!