Closed cyqsimon closed 2 weeks ago
Luap99
commented
3 weeks ago This seems like a very similar issue to https://github.com/containers/podman/issues/22192, so it likely needs to be fixed in pasta. But I'm not sure how to report directly to that project, so here's this issue.
https://passt.top/passt/about/#contribute
Regardless the maintainers are active here so this issue here is good enough.
cc @sbrivio-rh @dgibson
sbrivio-rh
commented
3 weeks ago 10.40.50.0/24 nhid 37 via 10.20.30.100 dev enp1s0 proto ospf metric 20
Weird, this is the first time I see a route with a nexthop identifier but a single nexthop (i.e. not multipath). On the other hand, it's been a while since the last time I played with OSPF.
@cyqsimon, would you be so kind as to run strace -e recvmsg ip -4 route show and report the RTM_NEWROUTE part here? I would need to see how the netlink message looks like in that case.
cyqsimon
commented
3 weeks ago @cyqsimon, would you be so kind as to run
strace -e recvmsg ip -4 route showand report theRTM_NEWROUTEpart here? I would need to see how the netlink message looks like in that case.
Of course. Here it is. I've replaced the addresses in the message to match my given example. Hope you find this okay.
recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=[[{nlmsg_len=60, nlmsg_type=RTM_NEWROUTE, nlmsg_flags=NLM_F_MULTI|NLM_F_DUMP_FILTERED, nlmsg_seq=1718111824, nlmsg_pid=87924}, {rtm_family=AF_INET, rtm_dst_len=0, rtm_src_len=0, rtm_tos=0, rtm_table=RT_TABLE_MAIN, rtm_protocol=RTPROT_STATIC, rtm_scope=RT_SCOPE_UNIVERSE, rtm_type=RTN_UNICAST, rtm_flags=0}, [[{nla_len=8, nla_type=RTA_TABLE}, RT_TABLE_MAIN], [{nla_len=8, nla_type=RTA_PRIORITY}, 100], [{nla_len=8, nla_type=RTA_GATEWAY}, inet_addr("10.20.30.1")], [{nla_len=8, nla_type=RTA_OIF}, if_nametoindex("enp1s0")]]], [{nlmsg_len=68, nlmsg_type=RTM_NEWROUTE, nlmsg_flags=NLM_F_MULTI|NLM_F_DUMP_FILTERED, nlmsg_seq=1718111824, nlmsg_pid=87924}, {rtm_family=AF_INET, rtm_dst_len=24, rtm_src_len=0, rtm_tos=0, rtm_table=RT_TABLE_MAIN, rtm_protocol=RTPROT_KERNEL, rtm_scope=RT_SCOPE_LINK, rtm_type=RTN_UNICAST, rtm_flags=0}, [[{nla_len=8, nla_type=RTA_TABLE}, RT_TABLE_MAIN], [{nla_len=8, nla_type=RTA_DST}, inet_addr("10.20.30.0")], [{nla_len=8, nla_type=RTA_PRIORITY}, 100], [{nla_len=8, nla_type=RTA_PREFSRC}, inet_addr("10.20.30.101")], [{nla_len=8, nla_type=RTA_OIF}, if_nametoindex("enp1s0")]]], [{nlmsg_len=76, nlmsg_type=RTM_NEWROUTE, nlmsg_flags=NLM_F_MULTI|NLM_F_DUMP_FILTERED, nlmsg_seq=1718111824, nlmsg_pid=87924}, {rtm_family=AF_INET, rtm_dst_len=24, rtm_src_len=0, rtm_tos=0, rtm_table=RT_TABLE_MAIN, rtm_protocol=RTPROT_OSPF, rtm_scope=RT_SCOPE_UNIVERSE, rtm_type=RTN_UNICAST, rtm_flags=0}, [[{nla_len=8, nla_type=RTA_TABLE}, RT_TABLE_MAIN], [{nla_len=8, nla_type=RTA_DST}, inet_addr("10.40.50.0")], [{nla_len=8, nla_type=RTA_PRIORITY}, 20], [{nla_len=8, nla_type=RTA_NH_ID}, "\x31\x00\x00\x00"], [{nla_len=8, nla_type=RTA_GATEWAY}, inet_addr("10.20.30.100")], [{nla_len=8, nla_type=RTA_OIF}, if_nametoindex("enp1s0")]]]], iov_len=32768}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 204@cyqsimon it took me a bit, but thanks to your recvmsg() dump I kind of reproduced this. I didn't really set up FRR with OSPF, but I hardcoded a similar route and it looks like the kernel rejects it because the nexthop identifier (RTA_NH_ID) is not valid in the target namespace. We should strip those attributes, because the target namespace will not have matching identifiers, in general.
To make sure this fixes your issue, could you please try this patch:
diff --git a/netlink.c b/netlink.c
index 4dbddb2..58822e9 100644
--- a/netlink.c
+++ b/netlink.c
@@ -608,6 +608,15 @@ int nl_route_dup(int s_src, unsigned int ifi_src,
* route invalid in the namespace. Strip off
* RTA_PREFSRC attributes to avoid that. */
rta->rta_type = RTA_UNSPEC;
+ } else if (rta->rta_type == RTA_NH_ID) {
+ /* Host routes set up via routing protocols
+ * (e.g. OSPF) might contain a nexthop ID (and
+ * not nexthop objects, which are taken care of
+ * in the RTA_MULTIPATH case above) that's not
+ * valid in the target namespace. Strip those as
+ * well.
+ */
+ rta->rta_type = RTA_UNSPEC;
}
}
and see if it fixes the issue for you? You don't have to use Podman or even install a new build. You can just git clone git://passt.top/passt, feed that to patch -p1 or apply manually, and build with make.
Then ./pasta --config-net (when an OSPF-derived route is present on the host) will show you if copying routes to the container now succeeds or not.
cyqsimon
commented
2 weeks ago @sbrivio-rh Yeah your patch fixes it. The self-compiled binary is able to successfully set up the namespace with the correct route now. Thanks for the great work!
I'll leave this issue open for now; please close when you see fit. Thanks again!
Luap99
commented
2 weeks ago sbrivio-rh
commented
2 weeks ago I'm preparing a release including this fix at the moment, by the way.
sbrivio-rh
commented
1 week ago Fixed in 2024_06_24.1ee2eca, and matching Fedora 40 update.
Issue Description
On a Linux system with OSPF routes,
podman start/runfails with the following error:Steps to reproduce the issue
This requires setting up OSPF which may be quite a lot of work, but here's the procedure anyways. Some useful links: Practical OSPF, FRR OSPFv2 user guide.
10.20.30.0/24.frrand configure it to distribute a route. For this example, its IP on the subnet is10.20.30.100and the distributed route is10.40.50.0/24.frrand configure it to receive the route. For this example, its IP on the subnet is10.20.30.101.podman run quay.io/podman/helloand observe the error.sudo systemctl stop frr.service && sudo ip route del 10.40.50.0/24to remove the OSPF route.Describe the results you received
N/A
Describe the results you expected
N/A
podman info output
Podman in a container
No
Privileged Or Rootless
Rootless
Upstream Latest Release
No
Additional environment details
No response
Additional information
This seems like a very similar issue to #22192, so it likely needs to be fixed in
pasta. But I'm not sure how to report directly to that project, so here's this issue.