search

edgecomllc / eupf

5G User Plane Function (UPF) based on eBPF
Apache License 2.0
100 stars 20 forks source link

Routing help needed #475

Closed Sabreu closed 10 months ago

Sabreu commented 10 months ago

Hi,

Any idea what kind of routing should be made in scenario like this:

openg5s core running in 10.10.1.1/16 eupf running in completely different pc 10.10.4.1/16 (build from source), same pc has also secondary ips 10.10.4.2-10.10.4.10 iperf3 servers running in the same pc as eupf (10.10.4.1).

UE and GNB are real ones, UE attach and PDN setup is OK.

With UE I can ping 10.10.4.1 but I can't see any ICMP packets with wireshark/tcpdump in 10.10.4.1 pc, I can't understand why... Ping to 10.10.4.2 or any other IP won't work, I can see ICMP coming from UE in this case, replies are not working. Iperf3 to 10.10.4.1 is not working either, I can see UE trying connection in wireshark but no answer from server.

UE IPs are from 10.45.0.1/24. GNB IP 10.10.10.41/16

I would like to not have any dedicated data network gateway, just use the same PC where eupf is running. I don't need any outside connection just ping/iperf3 from my own PC.

Thanks

Edit: How to get ICMP reply back to GNB as it seems like the PC(10.10.4.2) is asking who has via ARP? If "ip route add 10.45.0.0/24 via 10.10.10.41 dev enp1s0" is set then I can see 10.10.4.2 replying to ICMP and it's going to GNB but UE will not see the reply.

pirog-spb commented 10 months ago

Hi @Sabreu,

With UE I can ping 10.10.4.1 but I can't see any ICMP packets with wireshark/tcpdump in 10.10.4.1 pc, I can't understand why...

eUPF is based on xdp ebpf program. All packets interface xdp program attached to are processed earlier than tcpdump can catch them(except packets passing to kernel explicitly by xdp program).

eUPF by design can't forward packets to the same host where it works. This is because it's not possible to inject N6(downlink) packets from host to N3 tunnel via XDP.

In order to test iperf you have to deploy iperf server at least on a separate network interface within the same host (or on sepatare host).

If "ip route add 10.45.0.0/24 via 10.10.10.41 dev enp1s0" is set then I can see 10.10.4.2 replying to ICMP and it's going to GNB but UE will not see the reply.

It was a bug in eUPF until v0.5.2. eUPF didn't add gtp extension header with QCI to gtp packet. Some gNB ignored such response packets from eUPF. Please check that you used eUPF v0.5.2 for your tests.

-- BR, Alex

Sabreu commented 10 months ago

@pirog-spb Thanks a lot! yes that was the part we were missing (we were thinking that yesterday but I thought we could route it some other way easier). With 14900K 5Gbps (UDP) seems easy effort for eupf 👍 One thing seems odd to me, TCP DL (UE point of view) is only a few hundred kbps but TCP UL is working fine. Any idea what could cause that? With open5gs upf we have not seen this kind of issue.

Thanks

pirog-spb commented 10 months ago

One thing seems odd to me, TCP DL (UE point of view) is only a few hundred kbps but TCP UL is working fine. Any idea what could cause that? With open5gs upf we have not seen this kind of issue.

Could you share more details?

Common issue with iperf on veth interfaces is a wrong csum in DL packets. The solution is to disable checksum offloading on iperf network interface. Still not sure that it is relevant in your case but why not.

Sabreu commented 10 months ago

@pirog-spb Thanks a lot again, it was checksum offloading. TCP working nicely now 👍