networkop / cx

Containerised Cumulus VX
19 stars 4 forks source link

Netlink Errors for SVIs #7

Open jhlasnik opened 2 years ago

jhlasnik commented 2 years ago

Hello,

I'd like to report an issue I'm seeing on the 5.0.x images. I'm still working my way through learning the config so this certainly could be me missing something but wanted to report it as we discussed.

Full config and syslog files attached.

2022-06-20T20:24:44.955299+00:00 ee42457b760c9710 systemd[1]: nvue-startup.service: Succeeded.
2022-06-20T20:24:44.955449+00:00 ee42457b760c9710 systemd[1]: Started NVIDIA User Experience startup task.
2022-06-20T20:24:44.957667+00:00 ee42457b760c9710 systemd[1]: Started Cumulus Linux Switch Daemon.
2022-06-20T20:24:44.962302+00:00 ee42457b760c9710 systemd[1]: Starting Cumulus Linux Fast Interface Shutdown...
2022-06-20T20:24:44.965530+00:00 ee42457b760c9710 systemd[1]: Starting Cumulus Linux acltool...
2022-06-20T20:24:44.966862+00:00 ee42457b760c9710 systemd[1]: Cumulus Linux System Monitoring Daemon is not active.
2022-06-20T20:24:44.970732+00:00 ee42457b760c9710 systemd[1]: Dependency failed for Cumulus Linux LED Manager Daemon.
2022-06-20T20:24:44.972867+00:00 ee42457b760c9710 systemd[1]: ledmgrd.service: Job ledmgrd.service/start failed with result 'dependency'.
2022-06-20T20:24:44.975016+00:00 ee42457b760c9710 systemd[1]: Started Cumulus Linux Fan Control Daemon.
2022-06-20T20:24:44.985574+00:00 ee42457b760c9710 systemd[1]: Started Cumulus Linux ASIC Monitoring Daemon.
2022-06-20T20:24:44.998721+00:00 ee42457b760c9710 systemd[1]: Condition check resulted in Cumulus Linux Port Watch Event Daemon being skipped.
2022-06-20T20:24:44.999524+00:00 ee42457b760c9710 systemd[1]: Started Cumulus Linux Fast Interface Shutdown.
2022-06-20T20:24:44.999725+00:00 ee42457b760c9710 systemd[1]: acltool.service: Succeeded.
2022-06-20T20:24:45.001737+00:00 ee42457b760c9710 systemd[1]: Started Cumulus Linux acltool.
2022-06-20T20:24:45.009272+00:00 ee42457b760c9710 systemd[1]: Starting Network initialization...
2022-06-20T20:24:45.024598+00:00 ee42457b760c9710 networking[4043]: networking: Configuring network interfaces
2022-06-20T20:24:45.114619+00:00 ee42457b760c9710 : /usr/sbin/pwmd : : PWM Daemon doesn't manage fans on this platform
2022-06-20T20:24:45.150096+00:00 ee42457b760c9710 asic-monitor INFO: ASIC monitor is not supported on VX
2022-06-20T20:24:45.150495+00:00 ee42457b760c9710 systemd[1]: asic-monitor.service: Succeeded.
2022-06-20T20:24:45.615111+00:00 ee42457b760c9710 systemd-udevd[3914]: Using default interface naming scheme 'v240'.
2022-06-20T20:24:45.775802+00:00 ee42457b760c9710 info: eth0: enabling syslog for dhcp configuration
2022-06-20T20:24:45.776542+00:00 ee42457b760c9710 info: executing ip -o addr show eth0
2022-06-20T20:24:46.782390+00:00 ee42457b760c9710 info: executing /bin/ip vrf exec mgmt /sbin/dhclient -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0
2022-06-20T20:24:46.869280+00:00 ee42457b760c9710 dhclient[4143]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 1
2022-06-20T20:24:46.870037+00:00 ee42457b760c9710 dhclient[4143]: DHCPOFFER of 172.17.0.2 from 0.0.0.0
2022-06-20T20:24:46.870253+00:00 ee42457b760c9710 dhclient[4143]: DHCPREQUEST for 172.17.0.2 on eth0 to 255.255.255.255 port 67
2022-06-20T20:24:46.870566+00:00 ee42457b760c9710 dhclient[4143]: DHCPACK of 172.17.0.2 from 0.0.0.0
2022-06-20T20:24:46.942809+00:00 ee42457b760c9710 dhclient[4143]: Timeout too large reducing to: 2147483646 (TIME_MAX - 1)
2022-06-20T20:24:46.943144+00:00 ee42457b760c9710 dhclient[4143]: bound to 172.17.0.2 -- renewal in 2147483648 seconds.
2022-06-20T20:24:46.944073+00:00 ee42457b760c9710 info: executing ip -o addr show eth0
2022-06-20T20:24:46.946745+00:00 ee42457b760c9710 info: eth0: dhclient: new address detected: 172.17.0.2/16
2022-06-20T20:24:46.980694+00:00 ee42457b760c9710 networking[4043]: error: bridge vni show failed .. cmd '/sbin/bridge -force -batch - [vni add dev vxlan48 vni 10
2022-06-20T20:24:46.980823+00:00 ee42457b760c9710 networking[4043]: vni add dev vxlan48 vni 20
2022-06-20T20:24:46.980882+00:00 ee42457b760c9710 networking[4043]: vni add dev vxlan48 vni 30
2022-06-20T20:24:46.981163+00:00 ee42457b760c9710 networking[4043]: vni add dev vxlan48 vni 40]' failed: returned 1 (RTNETLINK answers: Operation not supported
2022-06-20T20:24:46.981223+00:00 ee42457b760c9710 networking[4043]: Command failed -:1
2022-06-20T20:24:46.981279+00:00 ee42457b760c9710 networking[4043]: RTNETLINK answers: Operation not supported
2022-06-20T20:24:46.981333+00:00 ee42457b760c9710 networking[4043]: Command failed -:2
2022-06-20T20:24:46.981392+00:00 ee42457b760c9710 networking[4043]: RTNETLINK answers: Operation not supported
2022-06-20T20:24:46.981447+00:00 ee42457b760c9710 networking[4043]: Command failed -:3
2022-06-20T20:24:46.981503+00:00 ee42457b760c9710 networking[4043]: RTNETLINK answers: Operation not supported
2022-06-20T20:24:46.981560+00:00 ee42457b760c9710 networking[4043]: Command failed -:4
2022-06-20T20:24:46.981613+00:00 ee42457b760c9710 networking[4043]: )
2022-06-20T20:24:47.079212+00:00 ee42457b760c9710 kernel: [   27.733014] device swp4 entered promiscuous mode
2022-06-20T20:24:47.079233+00:00 ee42457b760c9710 kernel: [   27.733669] device swp3 entered promiscuous mode
2022-06-20T20:24:47.086925+00:00 ee42457b760c9710 kernel: [   27.740743] device swp5 entered promiscuous mode
2022-06-20T20:24:47.093924+00:00 ee42457b760c9710 kernel: [   27.748122] device swp6 entered promiscuous mode
2022-06-20T20:24:47.095915+00:00 ee42457b760c9710 kernel: [   27.750013] device vxlan48 entered promiscuous mode
2022-06-20T20:24:47.201058+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 131, in <module>
2022-06-20T20:24:47.201198+00:00 ee42457b760c9710 networking[4043]:     sys.exit(main())
2022-06-20T20:24:47.201257+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 119, in main
2022-06-20T20:24:47.201314+00:00 ee42457b760c9710 networking[4043]:     return stand_alone()
2022-06-20T20:24:47.201370+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 103, in stand_alone
2022-06-20T20:24:47.201428+00:00 ee42457b760c9710 networking[4043]:     status = ifupdown2.main()
2022-06-20T20:24:47.201486+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 77, in main
2022-06-20T20:24:47.201543+00:00 ee42457b760c9710 networking[4043]:     self.handlers.get(self.op)(self.args)
2022-06-20T20:24:47.201603+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 198, in run_up
2022-06-20T20:24:47.201661+00:00 ee42457b760c9710 networking[4043]:     skipupperifaces=args.skipupperifaces)
2022-06-20T20:24:47.201718+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1839, in up
2022-06-20T20:24:47.201775+00:00 ee42457b760c9710 networking[4043]:     else False)
2022-06-20T20:24:47.201833+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1565, in _sched_ifaces
2022-06-20T20:24:47.202254+00:00 ee42457b760c9710 networking[4043]:     sort=True if (sort or ifupdownflags.flags.CLASS) else False)
2022-06-20T20:24:47.202333+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 589, in sched_ifaces
2022-06-20T20:24:47.202397+00:00 ee42457b760c9710 networking[4043]:     followdependents=followdependents)
2022-06-20T20:24:47.202458+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.202517+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.202573+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.202628+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.202685+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.202750+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.202806+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.202861+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.203250+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.203392+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.203516+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 307, in run_iface_graph
2022-06-20T20:24:47.203651+00:00 ee42457b760c9710 networking[4043]:     cls.run_iface_list_ops(ifupdownobj, ifaceobjs, ops)
2022-06-20T20:24:47.203777+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 183, in run_iface_list_ops
2022-06-20T20:24:47.203961+00:00 ee42457b760c9710 networking[4043]:     '0') == '1' else None)
2022-06-20T20:24:47.204089+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 103, in run_iface_op
2022-06-20T20:24:47.204235+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifupdownobj.get_ifaceobjs)
2022-06-20T20:24:47.204362+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1414, in run
2022-06-20T20:24:47.204498+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifaceobj_getfunc)
2022-06-20T20:24:47.204624+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1059, in _pre_up
2022-06-20T20:24:47.204793+00:00 ee42457b760c9710 networking[4043]:     self.log_error('%s: %s' % (ifaceobj.name, str(e)), ifaceobj)
2022-06-20T20:24:47.204970+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdownaddons/modulebase.py", line 108, in log_error
2022-06-20T20:24:47.205132+00:00 ee42457b760c9710 networking[4043]:     traceback.print_stack()
2022-06-20T20:24:47.205274+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.205396+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2698, in link_set_address
2022-06-20T20:24:47.205993+00:00 ee42457b760c9710 networking[4043]:     result = self.tx_nlpacket_get_response_with_error(link)
2022-06-20T20:24:47.206123+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2277, in tx_nlpacket_get_response_with_error
2022-06-20T20:24:47.206233+00:00 ee42457b760c9710 networking[4043]:     raise Exception(error_str)
2022-06-20T20:24:47.206338+00:00 ee42457b760c9710 networking[4043]: Exception: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.206445+00:00 ee42457b760c9710 networking[4043]: During handling of the above exception, another exception occurred:
2022-06-20T20:24:47.206549+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.206650+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1057, in _pre_up
2022-06-20T20:24:47.207038+00:00 ee42457b760c9710 networking[4043]:     self._process_bridge(ifaceobj, True, self.process_hwaddress(ifaceobj))
2022-06-20T20:24:47.207149+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1102, in process_hwaddress
2022-06-20T20:24:47.208011+00:00 ee42457b760c9710 networking[4043]:     self.netlink.link_set_address(ifaceobj.name, hwaddress, hwaddress_int)
2022-06-20T20:24:47.208147+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2706, in link_set_address
2022-06-20T20:24:47.208257+00:00 ee42457b760c9710 networking[4043]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-20T20:24:47.208368+00:00 ee42457b760c9710 networking[4043]: lib.nlcache.NetlinkError: netlink: br_default: cannot set dev br_default address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.208485+00:00 ee42457b760c9710 networking[4043]: error: br_default: netlink: br_default: cannot set dev br_default address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.213543+00:00 ee42457b760c9710 kernel: [   27.866466] IPv6: ADDRCONF(NETDEV_UP): br_default: link is not ready
2022-06-20T20:24:47.236935+00:00 ee42457b760c9710 kernel: [   27.891220] 8021q: 802.1Q VLAN Support v1.8
2022-06-20T20:24:47.242998+00:00 ee42457b760c9710 systemd-udevd[3918]: Using default interface naming scheme 'v240'.
2022-06-20T20:24:47.378771+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 131, in <module>
2022-06-20T20:24:47.378872+00:00 ee42457b760c9710 networking[4043]:     sys.exit(main())
2022-06-20T20:24:47.379094+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 119, in main
2022-06-20T20:24:47.379143+00:00 ee42457b760c9710 networking[4043]:     return stand_alone()
2022-06-20T20:24:47.379188+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 103, in stand_alone
2022-06-20T20:24:47.379238+00:00 ee42457b760c9710 networking[4043]:     status = ifupdown2.main()
2022-06-20T20:24:47.379283+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 77, in main
2022-06-20T20:24:47.379327+00:00 ee42457b760c9710 networking[4043]:     self.handlers.get(self.op)(self.args)
2022-06-20T20:24:47.379371+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 198, in run_up
2022-06-20T20:24:47.379415+00:00 ee42457b760c9710 networking[4043]:     skipupperifaces=args.skipupperifaces)
2022-06-20T20:24:47.379459+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1839, in up
2022-06-20T20:24:47.379503+00:00 ee42457b760c9710 networking[4043]:     else False)
2022-06-20T20:24:47.379547+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1565, in _sched_ifaces
2022-06-20T20:24:47.379591+00:00 ee42457b760c9710 networking[4043]:     sort=True if (sort or ifupdownflags.flags.CLASS) else False)
2022-06-20T20:24:47.379637+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 589, in sched_ifaces
2022-06-20T20:24:47.379681+00:00 ee42457b760c9710 networking[4043]:     followdependents=followdependents)
2022-06-20T20:24:47.379724+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.379769+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.379813+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.379856+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.380235+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.380284+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.380328+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 307, in run_iface_graph
2022-06-20T20:24:47.380372+00:00 ee42457b760c9710 networking[4043]:     cls.run_iface_list_ops(ifupdownobj, ifaceobjs, ops)
2022-06-20T20:24:47.380419+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 183, in run_iface_list_ops
2022-06-20T20:24:47.380462+00:00 ee42457b760c9710 networking[4043]:     '0') == '1' else None)
2022-06-20T20:24:47.380506+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 103, in run_iface_op
2022-06-20T20:24:47.380549+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifupdownobj.get_ifaceobjs)
2022-06-20T20:24:47.380593+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1414, in run
2022-06-20T20:24:47.380638+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifaceobj_getfunc)
2022-06-20T20:24:47.380682+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1059, in _pre_up
2022-06-20T20:24:47.382293+00:00 ee42457b760c9710 networking[4043]:     self.log_error('%s: %s' % (ifaceobj.name, str(e)), ifaceobj)
2022-06-20T20:24:47.382452+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdownaddons/modulebase.py", line 108, in log_error
2022-06-20T20:24:47.382619+00:00 ee42457b760c9710 networking[4043]:     traceback.print_stack()
2022-06-20T20:24:47.382755+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.382981+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2698, in link_set_address
2022-06-20T20:24:47.384001+00:00 ee42457b760c9710 networking[4043]:     result = self.tx_nlpacket_get_response_with_error(link)
2022-06-20T20:24:47.384188+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2277, in tx_nlpacket_get_response_with_error
2022-06-20T20:24:47.384328+00:00 ee42457b760c9710 networking[4043]:     raise Exception(error_str)
2022-06-20T20:24:47.384456+00:00 ee42457b760c9710 networking[4043]: Exception: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.384610+00:00 ee42457b760c9710 networking[4043]: During handling of the above exception, another exception occurred:
2022-06-20T20:24:47.384779+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.385123+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1057, in _pre_up
2022-06-20T20:24:47.386050+00:00 ee42457b760c9710 networking[4043]:     self._process_bridge(ifaceobj, True, self.process_hwaddress(ifaceobj))
2022-06-20T20:24:47.386187+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1102, in process_hwaddress
2022-06-20T20:24:47.386321+00:00 ee42457b760c9710 networking[4043]:     self.netlink.link_set_address(ifaceobj.name, hwaddress, hwaddress_int)
2022-06-20T20:24:47.386472+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2706, in link_set_address
2022-06-20T20:24:47.386604+00:00 ee42457b760c9710 networking[4043]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-20T20:24:47.386742+00:00 ee42457b760c9710 networking[4043]: lib.nlcache.NetlinkError: netlink: vlan10: cannot set dev vlan10 address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.386890+00:00 ee42457b760c9710 networking[4043]: error: vlan10: netlink: vlan10: cannot set dev vlan10 address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.460807+00:00 ee42457b760c9710 kernel: [   28.112636] device br_default entered promiscuous mode
2022-06-20T20:24:47.511472+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 131, in <module>
2022-06-20T20:24:47.511638+00:00 ee42457b760c9710 networking[4043]:     sys.exit(main())
2022-06-20T20:24:47.511700+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 119, in main
2022-06-20T20:24:47.511756+00:00 ee42457b760c9710 networking[4043]:     return stand_alone()
2022-06-20T20:24:47.511827+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 103, in stand_alone
2022-06-20T20:24:47.512111+00:00 ee42457b760c9710 networking[4043]:     status = ifupdown2.main()
2022-06-20T20:24:47.512180+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 77, in main
2022-06-20T20:24:47.512236+00:00 ee42457b760c9710 networking[4043]:     self.handlers.get(self.op)(self.args)
2022-06-20T20:24:47.512293+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 198, in run_up
2022-06-20T20:24:47.512349+00:00 ee42457b760c9710 networking[4043]:     skipupperifaces=args.skipupperifaces)
2022-06-20T20:24:47.512408+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1839, in up
2022-06-20T20:24:47.512464+00:00 ee42457b760c9710 networking[4043]:     else False)
2022-06-20T20:24:47.512516+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1565, in _sched_ifaces
2022-06-20T20:24:47.512571+00:00 ee42457b760c9710 networking[4043]:     sort=True if (sort or ifupdownflags.flags.CLASS) else False)
2022-06-20T20:24:47.512627+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 589, in sched_ifaces
2022-06-20T20:24:47.512694+00:00 ee42457b760c9710 networking[4043]:     followdependents=followdependents)
2022-06-20T20:24:47.512762+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.512816+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.512867+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.513387+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.513449+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.513506+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.513563+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 307, in run_iface_graph
2022-06-20T20:24:47.513617+00:00 ee42457b760c9710 networking[4043]:     cls.run_iface_list_ops(ifupdownobj, ifaceobjs, ops)
2022-06-20T20:24:47.513668+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 183, in run_iface_list_ops
2022-06-20T20:24:47.513748+00:00 ee42457b760c9710 networking[4043]:     '0') == '1' else None)
2022-06-20T20:24:47.514114+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 103, in run_iface_op
2022-06-20T20:24:47.514246+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifupdownobj.get_ifaceobjs)
2022-06-20T20:24:47.514374+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1414, in run
2022-06-20T20:24:47.514500+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifaceobj_getfunc)
2022-06-20T20:24:47.514635+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1059, in _pre_up
2022-06-20T20:24:47.514770+00:00 ee42457b760c9710 networking[4043]:     self.log_error('%s: %s' % (ifaceobj.name, str(e)), ifaceobj)
2022-06-20T20:24:47.514953+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdownaddons/modulebase.py", line 108, in log_error
2022-06-20T20:24:47.515086+00:00 ee42457b760c9710 networking[4043]:     traceback.print_stack()
2022-06-20T20:24:47.515211+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.515340+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2698, in link_set_address
2022-06-20T20:24:47.515493+00:00 ee42457b760c9710 networking[4043]:     result = self.tx_nlpacket_get_response_with_error(link)
2022-06-20T20:24:47.515646+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2277, in tx_nlpacket_get_response_with_error
2022-06-20T20:24:47.515778+00:00 ee42457b760c9710 networking[4043]:     raise Exception(error_str)
2022-06-20T20:24:47.521090+00:00 ee42457b760c9710 networking[4043]: Exception: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.521411+00:00 ee42457b760c9710 networking[4043]: During handling of the above exception, another exception occurred:
2022-06-20T20:24:47.521544+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.521669+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1057, in _pre_up
2022-06-20T20:24:47.521987+00:00 ee42457b760c9710 networking[4043]:     self._process_bridge(ifaceobj, True, self.process_hwaddress(ifaceobj))
2022-06-20T20:24:47.522120+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1102, in process_hwaddress
2022-06-20T20:24:47.525501+00:00 ee42457b760c9710 networking[4043]:     self.netlink.link_set_address(ifaceobj.name, hwaddress, hwaddress_int)
2022-06-20T20:24:47.526345+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2706, in link_set_address
2022-06-20T20:24:47.526499+00:00 ee42457b760c9710 networking[4043]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-20T20:24:47.527021+00:00 ee42457b760c9710 networking[4043]: lib.nlcache.NetlinkError: netlink: vlan30: cannot set dev vlan30 address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.527198+00:00 ee42457b760c9710 networking[4043]: error: vlan30: netlink: vlan30: cannot set dev vlan30 address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.634405+00:00 ee42457b760c9710 mstpd: create_br: Add bridge br_l3vni
2022-06-20T20:24:47.644108+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 131, in <module>
2022-06-20T20:24:47.644231+00:00 ee42457b760c9710 networking[4043]:     sys.exit(main())
2022-06-20T20:24:47.644324+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 119, in main
2022-06-20T20:24:47.644382+00:00 ee42457b760c9710 networking[4043]:     return stand_alone()
2022-06-20T20:24:47.644438+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 103, in stand_alone
2022-06-20T20:24:47.644493+00:00 ee42457b760c9710 networking[4043]:     status = ifupdown2.main()
2022-06-20T20:24:47.644545+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 77, in main
2022-06-20T20:24:47.644600+00:00 ee42457b760c9710 networking[4043]:     self.handlers.get(self.op)(self.args)
2022-06-20T20:24:47.644671+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 198, in run_up
2022-06-20T20:24:47.644742+00:00 ee42457b760c9710 networking[4043]:     skipupperifaces=args.skipupperifaces)
2022-06-20T20:24:47.644818+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1839, in up
2022-06-20T20:24:47.645326+00:00 ee42457b760c9710 networking[4043]:     else False)
2022-06-20T20:24:47.645425+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1565, in _sched_ifaces
2022-06-20T20:24:47.645490+00:00 ee42457b760c9710 networking[4043]:     sort=True if (sort or ifupdownflags.flags.CLASS) else False)
2022-06-20T20:24:47.645551+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 589, in sched_ifaces
2022-06-20T20:24:47.645606+00:00 ee42457b760c9710 networking[4043]:     followdependents=followdependents)
2022-06-20T20:24:47.645665+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.645722+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.645780+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.645835+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.646184+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.646258+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.646311+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.646385+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.646440+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.646492+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.646545+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 307, in run_iface_graph
2022-06-20T20:24:47.646598+00:00 ee42457b760c9710 networking[4043]:     cls.run_iface_list_ops(ifupdownobj, ifaceobjs, ops)
2022-06-20T20:24:47.646651+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 183, in run_iface_list_ops
2022-06-20T20:24:47.646706+00:00 ee42457b760c9710 networking[4043]:     '0') == '1' else None)
2022-06-20T20:24:47.646766+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 103, in run_iface_op
2022-06-20T20:24:47.646819+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifupdownobj.get_ifaceobjs)
2022-06-20T20:24:47.646879+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/bridge.py", line 4097, in run
2022-06-20T20:24:47.647380+00:00 ee42457b760c9710 networking[4043]:     op_handler(self, ifaceobj, ifaceobj_getfunc=ifaceobj_getfunc)
2022-06-20T20:24:47.647529+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/bridge.py", line 2845, in _up
2022-06-20T20:24:47.647668+00:00 ee42457b760c9710 networking[4043]:     self.up_bridge(ifaceobj, ifaceobj_getfunc)
2022-06-20T20:24:47.647801+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/bridge.py", line 2669, in up_bridge
2022-06-20T20:24:47.648019+00:00 ee42457b760c9710 networking[4043]:     newly_enslaved_ports = self._add_ports(ifaceobj, ifaceobj_getfunc)
2022-06-20T20:24:47.648156+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/bridge.py", line 1213, in _add_ports
2022-06-20T20:24:47.648287+00:00 ee42457b760c9710 networking[4043]:     %(ifaceobj.name, bridgeport), ifaceobj)
2022-06-20T20:24:47.648416+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdownaddons/modulebase.py", line 108, in log_error
2022-06-20T20:24:47.653329+00:00 ee42457b760c9710 networking[4043]:     traceback.print_stack()
2022-06-20T20:24:47.653575+00:00 ee42457b760c9710 networking[4043]: NoneType: None
2022-06-20T20:24:47.653718+00:00 ee42457b760c9710 networking[4043]: error: br_l3vni: bridge port vxlan99 does not exist
2022-06-20T20:24:47.679662+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 131, in <module>
2022-06-20T20:24:47.679783+00:00 ee42457b760c9710 networking[4043]:     sys.exit(main())
2022-06-20T20:24:47.679843+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 119, in main
2022-06-20T20:24:47.680132+00:00 ee42457b760c9710 networking[4043]:     return stand_alone()
2022-06-20T20:24:47.680202+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 103, in stand_alone
2022-06-20T20:24:47.680259+00:00 ee42457b760c9710 networking[4043]:     status = ifupdown2.main()
2022-06-20T20:24:47.680317+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 77, in main
2022-06-20T20:24:47.680402+00:00 ee42457b760c9710 networking[4043]:     self.handlers.get(self.op)(self.args)
2022-06-20T20:24:47.680460+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 198, in run_up
2022-06-20T20:24:47.680517+00:00 ee42457b760c9710 networking[4043]:     skipupperifaces=args.skipupperifaces)
2022-06-20T20:24:47.680574+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1839, in up
2022-06-20T20:24:47.680631+00:00 ee42457b760c9710 networking[4043]:     else False)
2022-06-20T20:24:47.680689+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1565, in _sched_ifaces
2022-06-20T20:24:47.680780+00:00 ee42457b760c9710 networking[4043]:     sort=True if (sort or ifupdownflags.flags.CLASS) else False)
2022-06-20T20:24:47.680847+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 589, in sched_ifaces
2022-06-20T20:24:47.681213+00:00 ee42457b760c9710 networking[4043]:     followdependents=followdependents)
2022-06-20T20:24:47.681279+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.681336+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.681412+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.681470+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.681527+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.681582+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.681638+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.681694+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.681750+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.681805+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.681860+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 307, in run_iface_graph
2022-06-20T20:24:47.682211+00:00 ee42457b760c9710 networking[4043]:     cls.run_iface_list_ops(ifupdownobj, ifaceobjs, ops)
2022-06-20T20:24:47.682347+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 183, in run_iface_list_ops
2022-06-20T20:24:47.682494+00:00 ee42457b760c9710 networking[4043]:     '0') == '1' else None)
2022-06-20T20:24:47.682620+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 103, in run_iface_op
2022-06-20T20:24:47.682752+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifupdownobj.get_ifaceobjs)
2022-06-20T20:24:47.682880+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1414, in run
2022-06-20T20:24:47.683045+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifaceobj_getfunc)
2022-06-20T20:24:47.683171+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1059, in _pre_up
2022-06-20T20:24:47.683300+00:00 ee42457b760c9710 networking[4043]:     self.log_error('%s: %s' % (ifaceobj.name, str(e)), ifaceobj)
2022-06-20T20:24:47.683430+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdownaddons/modulebase.py", line 108, in log_error
2022-06-20T20:24:47.683589+00:00 ee42457b760c9710 networking[4043]:     traceback.print_stack()
2022-06-20T20:24:47.683712+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.683852+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2698, in link_set_address
2022-06-20T20:24:47.684013+00:00 ee42457b760c9710 networking[4043]:     result = self.tx_nlpacket_get_response_with_error(link)
2022-06-20T20:24:47.685014+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2277, in tx_nlpacket_get_response_with_error
2022-06-20T20:24:47.685180+00:00 ee42457b760c9710 networking[4043]:     raise Exception(error_str)
2022-06-20T20:24:47.685312+00:00 ee42457b760c9710 networking[4043]: Exception: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.685449+00:00 ee42457b760c9710 networking[4043]: During handling of the above exception, another exception occurred:
2022-06-20T20:24:47.685573+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.685696+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1057, in _pre_up
2022-06-20T20:24:47.685844+00:00 ee42457b760c9710 networking[4043]:     self._process_bridge(ifaceobj, True, self.process_hwaddress(ifaceobj))
2022-06-20T20:24:47.686001+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1102, in process_hwaddress
2022-06-20T20:24:47.686134+00:00 ee42457b760c9710 networking[4043]:     self.netlink.link_set_address(ifaceobj.name, hwaddress, hwaddress_int)
2022-06-20T20:24:47.688946+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2706, in link_set_address
2022-06-20T20:24:47.689163+00:00 ee42457b760c9710 networking[4043]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-20T20:24:47.689315+00:00 ee42457b760c9710 networking[4043]: lib.nlcache.NetlinkError: netlink: br_l3vni: cannot set dev br_l3vni address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.689455+00:00 ee42457b760c9710 networking[4043]: error: br_l3vni: netlink: br_l3vni: cannot set dev br_l3vni address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.689618+00:00 ee42457b760c9710 mstpd: set_br_up: Set bridge br_l3vni up
2022-06-20T20:24:47.869023+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 131, in <module>
2022-06-20T20:24:47.869144+00:00 ee42457b760c9710 networking[4043]:     sys.exit(main())
2022-06-20T20:24:47.869203+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 119, in main
2022-06-20T20:24:47.869261+00:00 ee42457b760c9710 networking[4043]:     return stand_alone()
2022-06-20T20:24:47.869319+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 103, in stand_alone
2022-06-20T20:24:47.869377+00:00 ee42457b760c9710 networking[4043]:     status = ifupdown2.main()
2022-06-20T20:24:47.869436+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 77, in main
2022-06-20T20:24:47.869493+00:00 ee42457b760c9710 networking[4043]:     self.handlers.get(self.op)(self.args)
2022-06-20T20:24:47.869581+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 198, in run_up
2022-06-20T20:24:47.869640+00:00 ee42457b760c9710 networking[4043]:     skipupperifaces=args.skipupperifaces)
2022-06-20T20:24:47.869697+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1839, in up
2022-06-20T20:24:47.869752+00:00 ee42457b760c9710 networking[4043]:     else False)
2022-06-20T20:24:47.869808+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1565, in _sched_ifaces
2022-06-20T20:24:47.869864+00:00 ee42457b760c9710 networking[4043]:     sort=True if (sort or ifupdownflags.flags.CLASS) else False)
2022-06-20T20:24:47.870464+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 589, in sched_ifaces
2022-06-20T20:24:47.870526+00:00 ee42457b760c9710 networking[4043]:     followdependents=followdependents)
2022-06-20T20:24:47.870599+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.870657+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.870714+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.870770+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.870827+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.870881+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.871153+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 307, in run_iface_graph
2022-06-20T20:24:47.871289+00:00 ee42457b760c9710 networking[4043]:     cls.run_iface_list_ops(ifupdownobj, ifaceobjs, ops)
2022-06-20T20:24:47.871419+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 183, in run_iface_list_ops
2022-06-20T20:24:47.871570+00:00 ee42457b760c9710 networking[4043]:     '0') == '1' else None)
2022-06-20T20:24:47.871696+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 103, in run_iface_op
2022-06-20T20:24:47.871829+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifupdownobj.get_ifaceobjs)
2022-06-20T20:24:47.872000+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1414, in run
2022-06-20T20:24:47.872152+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifaceobj_getfunc)
2022-06-20T20:24:47.872279+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1059, in _pre_up
2022-06-20T20:24:47.872410+00:00 ee42457b760c9710 networking[4043]:     self.log_error('%s: %s' % (ifaceobj.name, str(e)), ifaceobj)
2022-06-20T20:24:47.872540+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdownaddons/modulebase.py", line 108, in log_error
2022-06-20T20:24:47.872672+00:00 ee42457b760c9710 networking[4043]:     traceback.print_stack()
2022-06-20T20:24:47.872827+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.873047+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2698, in link_set_address
2022-06-20T20:24:47.873185+00:00 ee42457b760c9710 networking[4043]:     result = self.tx_nlpacket_get_response_with_error(link)
2022-06-20T20:24:47.873342+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2277, in tx_nlpacket_get_response_with_error
2022-06-20T20:24:47.874012+00:00 ee42457b760c9710 networking[4043]:     raise Exception(error_str)
2022-06-20T20:24:47.874149+00:00 ee42457b760c9710 networking[4043]: Exception: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.874283+00:00 ee42457b760c9710 networking[4043]: During handling of the above exception, another exception occurred:
2022-06-20T20:24:47.874420+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.874547+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1057, in _pre_up
2022-06-20T20:24:47.874678+00:00 ee42457b760c9710 networking[4043]:     self._process_bridge(ifaceobj, True, self.process_hwaddress(ifaceobj))
2022-06-20T20:24:47.874988+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1102, in process_hwaddress
2022-06-20T20:24:47.875122+00:00 ee42457b760c9710 networking[4043]:     self.netlink.link_set_address(ifaceobj.name, hwaddress, hwaddress_int)
2022-06-20T20:24:47.876005+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2706, in link_set_address
2022-06-20T20:24:47.876154+00:00 ee42457b760c9710 networking[4043]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-20T20:24:47.876290+00:00 ee42457b760c9710 networking[4043]: lib.nlcache.NetlinkError: netlink: vlan20: cannot set dev vlan20 address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.876439+00:00 ee42457b760c9710 networking[4043]: error: vlan20: netlink: vlan20: cannot set dev vlan20 address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.975086+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 131, in <module>
2022-06-20T20:24:47.975199+00:00 ee42457b760c9710 networking[4043]:     sys.exit(main())
2022-06-20T20:24:47.975249+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 119, in main
2022-06-20T20:24:47.975299+00:00 ee42457b760c9710 networking[4043]:     return stand_alone()
2022-06-20T20:24:47.975347+00:00 ee42457b760c9710 networking[4043]:   File "/sbin/ifup", line 103, in stand_alone
2022-06-20T20:24:47.975396+00:00 ee42457b760c9710 networking[4043]:     status = ifupdown2.main()
2022-06-20T20:24:47.975446+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 77, in main
2022-06-20T20:24:47.975495+00:00 ee42457b760c9710 networking[4043]:     self.handlers.get(self.op)(self.args)
2022-06-20T20:24:47.975543+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/main.py", line 198, in run_up
2022-06-20T20:24:47.975591+00:00 ee42457b760c9710 networking[4043]:     skipupperifaces=args.skipupperifaces)
2022-06-20T20:24:47.975641+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1839, in up
2022-06-20T20:24:47.975689+00:00 ee42457b760c9710 networking[4043]:     else False)
2022-06-20T20:24:47.975737+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/ifupdownmain.py", line 1565, in _sched_ifaces
2022-06-20T20:24:47.975785+00:00 ee42457b760c9710 networking[4043]:     sort=True if (sort or ifupdownflags.flags.CLASS) else False)
2022-06-20T20:24:47.975833+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 589, in sched_ifaces
2022-06-20T20:24:47.975880+00:00 ee42457b760c9710 networking[4043]:     followdependents=followdependents)
2022-06-20T20:24:47.976592+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.976644+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.976693+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 297, in run_iface_graph
2022-06-20T20:24:47.976814+00:00 ee42457b760c9710 networking[4043]:     continueonfailure=False)
2022-06-20T20:24:47.976877+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 318, in run_iface_list
2022-06-20T20:24:47.977155+00:00 ee42457b760c9710 networking[4043]:     order, followdependents)
2022-06-20T20:24:47.977218+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 307, in run_iface_graph
2022-06-20T20:24:47.977275+00:00 ee42457b760c9710 networking[4043]:     cls.run_iface_list_ops(ifupdownobj, ifaceobjs, ops)
2022-06-20T20:24:47.977333+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 183, in run_iface_list_ops
2022-06-20T20:24:47.977391+00:00 ee42457b760c9710 networking[4043]:     '0') == '1' else None)
2022-06-20T20:24:47.977451+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdown/scheduler.py", line 103, in run_iface_op
2022-06-20T20:24:47.977509+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifupdownobj.get_ifaceobjs)
2022-06-20T20:24:47.977566+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1414, in run
2022-06-20T20:24:47.977622+00:00 ee42457b760c9710 networking[4043]:     ifaceobj_getfunc=ifaceobj_getfunc)
2022-06-20T20:24:47.977676+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1059, in _pre_up
2022-06-20T20:24:47.977731+00:00 ee42457b760c9710 networking[4043]:     self.log_error('%s: %s' % (ifaceobj.name, str(e)), ifaceobj)
2022-06-20T20:24:47.977789+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/ifupdownaddons/modulebase.py", line 108, in log_error
2022-06-20T20:24:47.977842+00:00 ee42457b760c9710 networking[4043]:     traceback.print_stack()
2022-06-20T20:24:47.978217+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.978362+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2698, in link_set_address
2022-06-20T20:24:47.978515+00:00 ee42457b760c9710 networking[4043]:     result = self.tx_nlpacket_get_response_with_error(link)
2022-06-20T20:24:47.978719+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2277, in tx_nlpacket_get_response_with_error
2022-06-20T20:24:47.978874+00:00 ee42457b760c9710 networking[4043]:     raise Exception(error_str)
2022-06-20T20:24:47.979038+00:00 ee42457b760c9710 networking[4043]: Exception: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.979186+00:00 ee42457b760c9710 networking[4043]: During handling of the above exception, another exception occurred:
2022-06-20T20:24:47.979328+00:00 ee42457b760c9710 networking[4043]: Traceback (most recent call last):
2022-06-20T20:24:47.979466+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1057, in _pre_up
2022-06-20T20:24:47.979626+00:00 ee42457b760c9710 networking[4043]:     self._process_bridge(ifaceobj, True, self.process_hwaddress(ifaceobj))
2022-06-20T20:24:47.979774+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/addons/address.py", line 1102, in process_hwaddress
2022-06-20T20:24:47.981474+00:00 ee42457b760c9710 networking[4043]:     self.netlink.link_set_address(ifaceobj.name, hwaddress, hwaddress_int)
2022-06-20T20:24:47.981691+00:00 ee42457b760c9710 networking[4043]:   File "/usr/share/ifupdown2/lib/nlcache.py", line 2706, in link_set_address
2022-06-20T20:24:47.981836+00:00 ee42457b760c9710 networking[4043]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-20T20:24:47.982008+00:00 ee42457b760c9710 networking[4043]: lib.nlcache.NetlinkError: netlink: vlan40: cannot set dev vlan40 address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:47.982169+00:00 ee42457b760c9710 networking[4043]: error: vlan40: netlink: vlan40: cannot set dev vlan40 address 93:4a:72:4d:0e:07: operation failed with 'Cannot assign requested address' (99)
2022-06-20T20:24:48.089120+00:00 ee42457b760c9710 kernel: [   28.740282] IPv6: ADDRCONF(NETDEV_CHANGE): br_default: link becomes ready
2022-06-20T20:24:48.106295+00:00 ee42457b760c9710 kernel: [   28.757333] IPv6: ADDRCONF(NETDEV_UP): vlan1298_l3: link is not ready
2022-06-20T20:24:48.106332+00:00 ee42457b760c9710 kernel: [   28.759372] IPv6: ADDRCONF(NETDEV_CHANGE): vlan1298_l3: link becomes ready

If you need any more information please let me know! syslog.log config.txt

networkop commented 2 years ago

I think that kernel version got updated but they didn't bump the version number, so I didn't detect it. I've build the new kernel image from the kernel extracted from 5.0 image: networkop/cl-kernel:5.0.1 Can you try specifying kernel: docker.io/networkop/cl-kernel:5.0.1 and see if you still see any errors?

networkop commented 2 years ago

btw, I've discovered another bug that manifests itself when you don't set the system-mac explicitly. If you're seeing an error when ifreload tries to set a system mac, try using networkop/cx:5.0.4

jhlasnik commented 2 years ago

Yeah I saw a similar issue when I had it set, I was trying a bunch of different combinations for this Inter-op lab with Arista.

I’ll try it out tonight and report back! Thanks!

Best regards, Joe Hlasnik


From: Michael Kashin @.> Sent: Tuesday, June 21, 2022 12:11:07 PM To: networkop/cx @.> Cc: Hlasnik, Joseph @.>; Author @.> Subject: Re: [networkop/cx] Netlink Errors for SVIs (Issue #7)

[ CAUTION: External Email ]

btw, I've discovered another bug that manifests itself when you don't set the system-mac explicitly. If you're seeing an error when ifreload tries to set a system mac, try using networkop/cx:5.0.4

— Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnetworkop%2Fcx%2Fissues%2F7%23issuecomment-1161964775&data=05%7C01%7Cjoseph.hlasnik%40expedient.com%7C51ee2775c545479e80d008da53a0a6f3%7Cdaaf576f055447d487baf848642811e0%7C0%7C0%7C637914246713581570%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jx77mP5TxQ1BHfjDZYnqrZEODNLXwRVDUfXbFD2z2gg%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAF5NS4PHOD32YGAT6SGURGDVQHSRXANCNFSM5ZKBVGXQ&data=05%7C01%7Cjoseph.hlasnik%40expedient.com%7C51ee2775c545479e80d008da53a0a6f3%7Cdaaf576f055447d487baf848642811e0%7C0%7C0%7C637914246713581570%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=E7%2Bmi%2FegAUAMXALXMIhFIFySLQrWvqI2z6OcTcFNcPg%3D&reserved=0. You are receiving this because you authored the thread.Message ID: @.***>

jhlasnik commented 2 years ago

So I did test setting the kernel on 5.0.1 and 5.0.4 and I'm still seeing those same errors on bootup with the system mac set.

root@clab-cx-sw1:mgmt:~# more /var/log/syslog | grep -E NetlinkError
2022-06-22T13:20:55.331706+00:00 1d58c2642a4b6fbd networking[4057]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-22T13:20:55.331849+00:00 1d58c2642a4b6fbd networking[4057]: lib.nlcache.NetlinkError: netlink: br_default: cannot set dev br_default address 89:b8:73:18:ad:e3: operation failed with 'Cannot assign requested address' (99)
2022-06-22T13:20:55.497630+00:00 1d58c2642a4b6fbd networking[4057]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-22T13:20:55.497739+00:00 1d58c2642a4b6fbd networking[4057]: lib.nlcache.NetlinkError: netlink: vlan10: cannot set dev vlan10 address 89:b8:73:18:ad:e3: operation failed with 'Cannot assign requested address' (99)
2022-06-22T13:20:55.636432+00:00 1d58c2642a4b6fbd networking[4057]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-22T13:20:55.636611+00:00 1d58c2642a4b6fbd networking[4057]: lib.nlcache.NetlinkError: netlink: vlan30: cannot set dev vlan30 address 89:b8:73:18:ad:e3: operation failed with 'Cannot assign requested address' (99)
2022-06-22T13:20:55.882610+00:00 1d58c2642a4b6fbd networking[4057]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-22T13:20:55.882778+00:00 1d58c2642a4b6fbd networking[4057]: lib.nlcache.NetlinkError: netlink: br_l3vni: cannot set dev br_l3vni address 89:b8:73:18:ad:e3: operation failed with 'Cannot assign requested address' (99)
2022-06-22T13:20:56.126300+00:00 1d58c2642a4b6fbd networking[4057]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-22T13:20:56.126508+00:00 1d58c2642a4b6fbd networking[4057]: lib.nlcache.NetlinkError: netlink: vlan20: cannot set dev vlan20 address 89:b8:73:18:ad:e3: operation failed with 'Cannot assign requested address' (99)
2022-06-22T13:20:56.278890+00:00 1d58c2642a4b6fbd networking[4057]:     raise NetlinkError(e, "cannot set dev %s address %s" % (ifname, hw_address), ifname=ifname)
2022-06-22T13:20:56.279057+00:00 1d58c2642a4b6fbd networking[4057]: lib.nlcache.NetlinkError: netlink: vlan40: cannot set dev vlan40 address 89:b8:73:18:ad:e3: operation failed with 'Cannot assign requested address' (99)
# topology documentation:
name: cx
topology:
  nodes:
    sw1:
      kind: cvx
      image: networkop/cx:5.0.4
      kernel: docker.io/networkop/cl-kernel:5.0.1
      binds:
        - hosts/clab-cx-sw1/startup.yaml:/etc/nvue.d/startup.yaml
    sw2:
      kind: cvx
      image: networkop/cx:5.0.4
      kernel: docker.io/networkop/cl-kernel:5.0.1
      binds:
        - hosts/clab-cx-sw2/startup.yaml:/etc/nvue.d/startup.yaml
networkop commented 2 years ago

can you share the SVG file you get using systemd-analyze plot > boot.svg?

jhlasnik commented 2 years ago

File attached

boot

networkop commented 2 years ago

it seems to work fine in my case. I think it could be down to the background CPU noise and race conditions. It will work if you manually do a systemctl restart sys-mac-shot. As you can see from this graph, I've tried to put it as close to the end as possible. The right way for me would be to try and locate how/why the wrong(multicast) MAC is being set in /run/system_mac and try to correct/patch that. However so far I couldn't find it. So what I'll do instead is release another version where I'll delay the sys-mac-shot execution by, say 10 seconds. This happens after network-online so it should delay the time to login.

networkop commented 2 years ago

networkop/cx:5.0.5 is out with the patch.

networkop commented 2 years ago

Looks like I've found where the MAC bug is coming from. https://github.com/networkop/cx/blob/5f8dc609a3a0c7f5d16c6cfc935100f49332ab09/hacks/decode-syseeprom.5.1#L373 and https://docs.python.org/3/library/uuid.html#uuid.getnode

If all attempts to obtain the hardware address fail, we choose a random 48-bit number with the multicast bit (least significant bit of the first octet) set to 1 as recommended in [RFC 4122](https://tools.ietf.org/html/rfc4122.html).

thanks to https://stackoverflow.com/a/159195

networkop commented 2 years ago

this should be fixed in networkop/cx:5.0.6