Add capability to block in netstack_create() until dumps are collected - Githubissues

dankamongmen / libnetstack

Small library around rtnetlink to track networking stack elements

https://nick-black.com/dankwiki/index.php/Libnetstack

Apache License 2.0

10 stars 0 forks source link

Add capability to block in netstack_create() until dumps are collected #17

Closed dankamongmen closed 1 year ago

dankamongmen commented 5 years ago

It would probably be useful to add a capability to netstack_create() that blocks until the dump requests have been responded to. Otherwise, a caller can get a successful return from netstack_create(), and yet be unable to access expected elements (and enumerations will see no elements).

In fact, this should probably be the default.

dankamongmen commented 5 years ago

NETSTACK_INITIAL_EVENTS_ASYNC: The default. Upon creation, objects will be enumerated, but netstack_create() will return after sending the necessary requests. Events might arrive before or after netstack_create() returns.
NETSTACK_INITIAL_EVENTS_BLOCK: Don't return from netstack_create() until all objects have been enumerated. If used, the cache may be safely interrogated once netstack_create() returns. Otherwise, existing objects might not show up for a short time.
NETSTACK_INITIAL_EVENTS_NONE: Don't perform the initial enumeration.

dankamongmen commented 5 years ago

I've implemented _BLOCK. _SYNC already works as the default. Added unit tests for _BLOCK and _NONE. _NONE fails as expected. Implementing _NONE now.

dankamongmen commented 5 years ago

Implemented and unit tested.

dankamongmen commented 1 year ago

This is still happening, to a degree. In my application, I'm calling with NETSTACK_INITIAL_EVENTS_BLOCK. I sometimes get:

discovered 9 addresses and 17 routes among 6 interfaces

and other times get:

discovered 10 addresses and 23 routes among 6 interfaces

despite no addresses or routes being changed. The behavior is nondeterministic.

dankamongmen commented 1 year ago

ok, this does not seem to be a race: careful strace analysis reveals that netstack-demo is getting 10 results, as expected; my application sometimes gets 10, and sometimes gets 9, missing the second result (100.113.22.255 on ixgbe1 is missing)...

dankamongmen commented 1 year ago

working:

[pid 1791943] <... recvmsg resumed>{msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[[{nlmsg_len=76, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=8, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("127.0.0.1")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("127.0.0.1")], [{nla_len=7, nla_type=IFA_LABEL}, "lo"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=1336, tstamp=1336}]]], [{nlmsg_len=96, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=0, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ixgbe1")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("100.113.22.12")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("100.113.22.12")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("100.113.22.255")], [{nla_len=11, nla_type=IFA_LABEL}, "ixgbe1"], [{nla_len=8, nla_type=IFA_FLAGS}, 0], [{nla_len=8, nla_type=IFA_RT_PRIORITY}, 1], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=28751, ifa_valid=28751, cstamp=13521660, tstamp=13521660}]]], [{nlmsg_len=88, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ax200")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("192.168.88.100")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("192.168.88.100")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("192.168.88.255")], [{nla_len=10, nla_type=IFA_LABEL}, "ax200"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5830, tstamp=5830}]]], [{nlmsg_len=80, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=26, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ax200")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.0.0.5")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.0.0.5")], [{nla_len=10, nla_type=IFA_LABEL}, "ax200"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=3068923, tstamp=3068923}]]], [{nlmsg_len=92, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.128.0.1")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.128.0.1")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("10.128.0.255")], [{nla_len=14, nla_type=IFA_LABEL}, "smsc75usb"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=2632, tstamp=2632}]]], [{nlmsg_len=84, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=26, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.0.0.11")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.0.0.11")], [{nla_len=14, nla_type=IFA_LABEL}, "smsc75usb"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=3072548, tstamp=3072548}]]], [{nlmsg_len=76, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("wg0")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("192.168.128.2")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("192.168.128.2")], [{nla_len=8, nla_type=IFA_LABEL}, "wg0"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5512, tstamp=5512}]]]], iov_len=16384}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, MSG_PEEK|MSG_TRUNC) = 592
[pid 1791943] <... recvmsg resumed>{msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[[{nlmsg_len=76, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=8, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("127.0.0.1")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("127.0.0.1")], [{nla_len=7, nla_type=IFA_LABEL}, "lo"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=1336, tstamp=1336}]]], [{nlmsg_len=96, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=0, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ixgbe1")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("100.113.22.12")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("100.113.22.12")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("100.113.22.255")], [{nla_len=11, nla_type=IFA_LABEL}, "ixgbe1"], [{nla_len=8, nla_type=IFA_FLAGS}, 0], [{nla_len=8, nla_type=IFA_RT_PRIORITY}, 1], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=28751, ifa_valid=28751, cstamp=13521660, tstamp=13521660}]]], [{nlmsg_len=88, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ax200")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("192.168.88.100")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("192.168.88.100")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("192.168.88.255")], [{nla_len=10, nla_type=IFA_LABEL}, "ax200"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5830, tstamp=5830}]]], [{nlmsg_len=80, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=26, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ax200")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.0.0.5")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.0.0.5")], [{nla_len=10, nla_type=IFA_LABEL}, "ax200"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=3068923, tstamp=3068923}]]], [{nlmsg_len=92, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.128.0.1")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.128.0.1")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("10.128.0.255")], [{nla_len=14, nla_type=IFA_LABEL}, "smsc75usb"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=2632, tstamp=2632}]]], [{nlmsg_len=84, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=26, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.0.0.11")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.0.0.11")], [{nla_len=14, nla_type=IFA_LABEL}, "smsc75usb"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=3072548, tstamp=3072548}]]], [{nlmsg_len=76, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("wg0")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("192.168.128.2")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("192.168.128.2")], [{nla_len=8, nla_type=IFA_LABEL}, "wg0"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5512, tstamp=5512}]]]], iov_len=16384}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 592
[pid 1791943] recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[[{nlmsg_len=72, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET6, ifa_prefixlen=128, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [[{nla_len=20, nla_type=IFA_ADDRESS}, inet_pton(AF_INET6, "::1")], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=1336, tstamp=1336}], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT]]], [{nlmsg_len=80, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET6, ifa_prefixlen=64, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_LINK, ifa_index=if_nametoindex("ax200")}, [[{nla_len=20, nla_type=IFA_ADDRESS}, inet_pton(AF_INET6, "fe80::3e9c:fff:fe4f:34e5")], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5828, tstamp=5828}], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=5, nla_type=IFA_PROTO}, "\x03"]]], [{nlmsg_len=80, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET6, ifa_prefixlen=64, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_LINK, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=20, nla_type=IFA_ADDRESS}, inet_pton(AF_INET6, "fe80::823f:5dff:fe08:159b")], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=2628, tstamp=2628}], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=5, nla_type=IFA_PROTO}, "\x03"]]]], iov_len=16384}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, MSG_PEEK|MSG_TRUNC) = 232
[pid 1791943] recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[[{nlmsg_len=72, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET6, ifa_prefixlen=128, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [[{nla_len=20, nla_type=IFA_ADDRESS}, inet_pton(AF_INET6, "::1")], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=1336, tstamp=1336}], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT]]], [{nlmsg_len=80, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET6, ifa_prefixlen=64, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_LINK, ifa_index=if_nametoindex("ax200")}, [[{nla_len=20, nla_type=IFA_ADDRESS}, inet_pton(AF_INET6, "fe80::3e9c:fff:fe4f:34e5")], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5828, tstamp=5828}], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=5, nla_type=IFA_PROTO}, "\x03"]]], [{nlmsg_len=80, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750478, nlmsg_pid=-2112137274}, {ifa_family=AF_INET6, ifa_prefixlen=64, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_LINK, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=20, nla_type=IFA_ADDRESS}, inet_pton(AF_INET6, "fe80::823f:5dff:fe08:159b")], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=2628, tstamp=2628}], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=5, nla_type=IFA_PROTO}, "\x03"]]]], iov_len=16384}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 232

nonworking:

[pid 1786376] <... recvmsg resumed>{msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[[{nlmsg_len=76, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=8, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("127.0.0.1")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("127.0.0.1")], [{nla_len=7, nla_type=IFA_LABEL}, "lo"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=1336, tstamp=1336}]]], [{nlmsg_len=88, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ax200")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("192.168.88.100")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("192.168.88.100")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("192.168.88.255")], [{nla_len=10, nla_type=IFA_LABEL}, "ax200"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5830, tstamp=5830}]]], [{nlmsg_len=80, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=26, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ax200")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.0.0.5")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.0.0.5")], [{nla_len=10, nla_type=IFA_LABEL}, "ax200"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=3068923, tstamp=3068923}]]], [{nlmsg_len=92, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.128.0.1")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.128.0.1")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("10.128.0.255")], [{nla_len=14, nla_type=IFA_LABEL}, "smsc75usb"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=2632, tstamp=2632}]]], [{nlmsg_len=84, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=26, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.0.0.11")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.0.0.11")], [{nla_len=14, nla_type=IFA_LABEL}, "smsc75usb"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=3072548, tstamp=3072548}]]], [{nlmsg_len=76, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("wg0")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("192.168.128.2")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("192.168.128.2")], [{nla_len=8, nla_type=IFA_LABEL}, "wg0"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5512, tstamp=5512}]]]], iov_len=16384}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, MSG_PEEK|MSG_TRUNC) = 496
[pid 1786376] <... recvmsg resumed>{msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[[{nlmsg_len=76, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=8, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_HOST, ifa_index=if_nametoindex("lo")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("127.0.0.1")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("127.0.0.1")], [{nla_len=7, nla_type=IFA_LABEL}, "lo"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=1336, tstamp=1336}]]], [{nlmsg_len=88, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ax200")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("192.168.88.100")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("192.168.88.100")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("192.168.88.255")], [{nla_len=10, nla_type=IFA_LABEL}, "ax200"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5830, tstamp=5830}]]], [{nlmsg_len=80, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=26, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("ax200")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.0.0.5")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.0.0.5")], [{nla_len=10, nla_type=IFA_LABEL}, "ax200"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=3068923, tstamp=3068923}]]], [{nlmsg_len=92, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.128.0.1")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.128.0.1")], [{nla_len=8, nla_type=IFA_BROADCAST}, inet_addr("10.128.0.255")], [{nla_len=14, nla_type=IFA_LABEL}, "smsc75usb"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=2632, tstamp=2632}]]], [{nlmsg_len=84, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=26, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("smsc75usb")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("10.0.0.11")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("10.0.0.11")], [{nla_len=14, nla_type=IFA_LABEL}, "smsc75usb"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=3072548, tstamp=3072548}]]], [{nlmsg_len=76, nlmsg_type=RTM_NEWADDR, nlmsg_flags=NLM_F_MULTI, nlmsg_seq=1676750425, nlmsg_pid=-1227144723}, {ifa_family=AF_INET, ifa_prefixlen=24, ifa_flags=IFA_F_PERMANENT, ifa_scope=RT_SCOPE_UNIVERSE, ifa_index=if_nametoindex("wg0")}, [[{nla_len=8, nla_type=IFA_ADDRESS}, inet_addr("192.168.128.2")], [{nla_len=8, nla_type=IFA_LOCAL}, inet_addr("192.168.128.2")], [{nla_len=8, nla_type=IFA_LABEL}, "wg0"], [{nla_len=8, nla_type=IFA_FLAGS}, IFA_F_PERMANENT], [{nla_len=20, nla_type=IFA_CACHEINFO}, {ifa_prefered=4294967295, ifa_valid=4294967295, cstamp=5512, tstamp=5512}]]]], iov_len=16384}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 496

dankamongmen commented 1 year ago

the sendmsg arguments appear to be exactly the same...

dankamongmen commented 1 year ago

recvmsg pattern:

both get an unfinished recvmsg() (blocking) both get a 2816-byte RTM_NEWLINK reply both get a second 2816-byte RTM_NEWLINK reply both get a 5672-byte RTM_NEWLINK reply both get a second 5672-byte RTM_NEWLINK reply both get an NLMSG_DONE both get a second NLMSG_DONE both get an unfinished recvmsg() (blocking) loser gets a 496-byte RTM_NEWADDR, winner gets a 592-byte RTM_NEWADDR. only the winner has our desired address.

wait.....you know what? this could be residue from the just-executed xdp-loader remove operation on the interface, wich i believe brings it down and back up. shit.

dankamongmen commented 1 year ago

yeah.

dankamongmen commented 1 year ago

yeah. here's netstack-demo running concurrently with the application, when an XDP program is left up before it runs...

Waiting on signal...
N*  3 [IPv4] 239.255.255.250 01:00:5e:7f:ff:fa
N*  3 [IPv4] 100.113.22.1 dc:2c:6e:90:3b:57
N*  3 [IPv4] 224.0.0.251 01:00:5e:00:00:fb
   3 [ixgbe1] ether 1 18:c0:4d:84:84:c6 mtu 1500 rxq 64 txq 64
   3 [ixgbe1] ether 1 18:c0:4d:84:84:c6 mtu 1500 rxq 64 txq 64
R*[IPv4] unicast 100.113.22.1 8.8.8.8/32 dhcp metric 0 prio 1 in 0 out 3
R*[IPv4] unicast 100.113.22.1 199.185.175.9/32 dhcp metric 0 prio 1 in 0 out 3
R*[IPv4] unicast 100.113.22.1/32 dhcp metric 0 prio 1 in 0 out 3
R*[IPv4] unicast 100.113.22.1 129.6.15.28/32 dhcp metric 0 prio 1 in 0 out 3
R*[IPv4] unicast 100.113.22.1 dhcp metric 0 prio 1 in 0 out 3
A*  3 [IPv4] 100.113.22.12/24
R*[IPv4] unicast 100.113.22.0/24 kernel metric 0 prio 1 in 0 out 3
R*[IPv4] broadcast 100.113.22.255/32 kernel metric 0 prio 0 in 0 out 3
R*[IPv4] local 100.113.22.12/32 kernel metric 0 prio 0 in 0 out 3
N*  3 [IPv4] 239.255.255.250 01:00:5e:7f:ff:fa
N*  3 [IPv4] 100.113.22.1