search

FRRouting / frr

The FRRouting Protocol Suite
https://frrouting.org/
Other
3.33k stars 1.25k forks source link

netlink-cmd (NS 0) error: Operation not supported and zebra Invalid address family: 128 received from kernel route change: 24 #4375

Closed melicherm closed 5 years ago

melicherm commented 5 years ago

Hi, getting all the time (like every second) this errors in the frr.log.

Could not find any hint what is causing this behavior.

The setup is easy (1 interface uplink - public network, 1 interface downlink - private network). PIM SM and IGMP v2 on both interfaces

May 21 01:17:59 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235404, pid=2372531164
May 21 01:17:59 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235405, pid=2372531164
May 21 01:17:59 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235406, pid=2372531164
May 21 01:18:00 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235407, pid=2372531164
May 21 01:18:00 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235408, pid=2372531164
May 21 01:18:00 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235409, pid=2372531164
May 21 01:18:00 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235410, pid=2372531164
May 21 01:18:00 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235411, pid=2372531164
May 21 01:18:01 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235412, pid=2372531164
May 21 01:18:01 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235413, pid=2372531164
May 21 01:18:01 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235414, pid=2372531164
May 21 01:18:01 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235415, pid=2372531164
May 21 01:18:01 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235416, pid=2372531164
May 21 01:18:01 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235417, pid=2372531164
May 21 01:18:02 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235418, pid=2372531164
May 21 01:18:02 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235419, pid=2372531164
May 21 01:18:02 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235420, pid=2372531164
May 21 01:18:02 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235421, pid=2372531164
May 21 01:18:02 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235422, pid=2372531164
May 21 01:18:02 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235423, pid=2372531164
May 21 01:18:02 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235424, pid=2372531164
May 21 01:18:02 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235425, pid=2372531164
May 21 01:18:02 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235426, pid=2372531164
May 21 01:18:03 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235427, pid=2372531164
May 21 01:18:03 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235428, pid=2372531164
May 21 01:18:03 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235429, pid=2372531164
May 21 01:18:03 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235430, pid=2372531164
May 21 01:18:03 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235431, pid=2372531164
May 21 01:18:03 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235432, pid=2372531164
May 21 01:18:04 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235433, pid=2372531164
May 21 01:18:04 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235434, pid=2372531164
May 21 01:18:04 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235435, pid=2372531164
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:04 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:05 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235436, pid=2372531164
May 21 01:18:05 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235437, pid=2372531164
May 21 01:18:05 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235438, pid=2372531164
May 21 01:18:05 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235439, pid=2372531164
May 21 01:18:05 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235440, pid=2372531164
May 21 01:18:06 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235441, pid=2372531164
May 21 01:18:06 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235442, pid=2372531164
May 21 01:18:06 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235443, pid=2372531164
May 21 01:18:06 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235444, pid=2372531164
May 21 01:18:06 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235445, pid=2372531164
May 21 01:18:06 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235446, pid=2372531164
May 21 01:18:06 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235447, pid=2372531164
May 21 01:18:06 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235448, pid=2372531164
May 21 01:18:07 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235449, pid=2372531164
May 21 01:18:07 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235450, pid=2372531164
May 21 01:18:07 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235451, pid=2372531164
May 21 01:18:07 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235452, pid=2372531164
May 21 01:18:07 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235453, pid=2372531164
May 21 01:18:08 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235454, pid=2372531164
May 21 01:18:08 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235455, pid=2372531164
May 21 01:18:08 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 25
May 21 01:18:08 iris1 zebra[29193]: Invalid address family: 128 received from kernel route change: 24
May 21 01:18:08 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235456, pid=2372531164
May 21 01:18:08 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235457, pid=2372531164
May 21 01:18:08 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235458, pid=2372531164
May 21 01:18:08 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235459, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235460, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235461, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235462, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235463, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235464, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235465, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235466, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235467, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235468, pid=2372531164
May 21 01:18:09 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235469, pid=2372531164
May 21 01:18:10 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235470, pid=2372531164
May 21 01:18:10 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235471, pid=2372531164
May 21 01:18:10 iris1 zebra[29193]: netlink-cmd (NS 0) error: Operation not supported, type=RTM_GETROUTE(26), seq=235472, pid=2372531164

Kernel: 4.10.8

FRR version: frr 6.0-1 debian9+1 amd64 BGP/OSPF/RIP/RIPng/ISIS/PIM/LDP routing daemon forked from Quagga frr-doc 6.0-1 debian9+1 all BGP/OSPF/RIP/RIPng/ISIS/PIM/LDP routing daemon (documentation) frr-pythontools 6.0-1 debian9+1 all BGP/OSPF/RIP/RIPng/ISIS/PIM/LDP routing daemon (Python Tools)

rwestphal commented 5 years ago

According to our Features and Kernel Support page, PIM requires Linux kernel >= 4.18. That could explain the ENOTSUP errors. @donaldsharp might know the details better.

melicherm commented 5 years ago

This week going to test the 4.19 backports kernel. Thought it could be a kernel issue, but i was not sure.

The router handles 10Gbit traffic of aprox. 700 users and up to 2000 devices, so i cannot kill the production one, but going to test / migrate to a test router soon.

BTW PIM and IGMP works like a charm... ~ 200Mbit of Multicast traffic, but the errors are annoying and representing some kind of issue.

Thank you @rwestphal ...

donaldsharp commented 5 years ago

Yes please try a later kernel. The operation not supported is coming from the request to the kernel to give us mroute stats so we can switch over to spt.

qlyoung commented 5 years ago

@polychaeta autoclose in 3 days

melicherm commented 5 years ago

Hi, testing it right now.

Still getting the same errors in the log. @donaldsharp ?

ii frr 6.0.3-1 amd64 FRRouting suite of internet protocols (BGP, OSPF, IS-IS, ...) ii frr-pythontools 6.0.3-1 all FRRouting suite - Python tools

Linux iris1 4.19.0-5-amd64 #1 SMP Debian 4.19.37-5 (2019-06-19) x86_64 GNU/Linux

Jul 9 21:23:52 iris1 pimd[3789]: Ignoring unsupported IGMP message type: 20 Jul 9 21:23:54 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24 Jul 9 21:23:54 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24 Jul 9 21:23:57 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 25 Jul 9 21:23:57 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 25 Jul 9 21:23:57 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24 Jul 9 21:23:57 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24 Jul 9 21:23:58 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 25 Jul 9 21:23:58 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24 Jul 9 21:23:58 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 25 Jul 9 21:23:59 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24 Jul 9 21:23:59 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 25 Jul 9 21:23:59 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 25 Jul 9 21:23:59 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 25 Jul 9 21:23:59 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 25 Jul 9 21:23:59 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24 Jul 9 21:23:59 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24 Jul 9 21:23:59 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24 Jul 9 21:23:59 iris1 zebra[967]: Invalid address family: 128 received from kernel route change: 24

ron5t commented 4 years ago

hello, any update on this? i get the same error but with an older 3.x kernel