search

edgecomllc / eupf

5G User Plane Function (UPF) based on eBPF
Apache License 2.0
101 stars 20 forks source link

eupf fib_lookup failed with cause 4 #117

Closed pirog-spb closed 1 year ago

pirog-spb commented 1 year ago
PapaySail commented 1 year ago

Problem diag:

root@ueransim-ue-7f76db59c9-pqpqk:/ueransim/build# ping 127.0.0.1 -I uesimtun0
PING 127.0.0.1 (127.0.0.1) from 10.1.0.1 uesimtun0: 56(84) bytes of data.
^C
--- 127.0.0.1 ping statistics ---
22 packets transmitted, 0 received, 100% packet loss, time 21502ms

At node:

sergo@edgecom:~$ sudo cat /sys/kernel/debug/tracing/trace_pipe

          nr-gnb-159158  [007] d.s11  6786.469274: bpf_trace_printk: upf_ip_entrypoint start
          nr-gnb-159158  [007] d.s11  6786.469293: bpf_trace_printk: upf: gtp-u received
          nr-gnb-159158  [007] d.s11  6786.469299: bpf_trace_printk: upf: gtp pdu [ 10.100.50.236 -> 10.100.50.233 ]
          nr-gnb-159158  [007] d.s11  6786.469303: bpf_trace_printk: upf: uplink session for teid:1 far:1 headrm:0
          nr-gnb-159158  [007] d.s11  6786.469306: bpf_trace_printk: upf: far for teid:1 far:1 action:2
          nr-gnb-159158  [007] d.s11  6786.469323: bpf_trace_printk: upf: bpf_fib_lookup 7: 10.1.0.1 -> 127.0.0.1
          nr-gnb-159158  [007] d.s11  6786.469324: bpf_trace_printk: upf: bpf_fib_lookup nexthop: 10.100.50.238
          nr-gnb-159158  [007] d.s11  6787.488647: bpf_trace_printk: upf_ip_entrypoint start
ℹ bpf_fib_lookup BPF_FIB_LKUP_RET_ codes

[Return](https://man7.org/linux/man-pages/man7/bpf-helpers.7.html) • < 0 if any input argument is invalid • 0 on success (packet is forwarded, nexthop neighbor exists) • > 0 one of BPF_FIB_LKUP_RET_ codes explaining why the packet is not forwarded or needs assist from full stack ```diff +enum { + BPF_FIB_LKUP_RET_SUCCESS, /* lookup successful */ + BPF_FIB_LKUP_RET_BLACKHOLE, /* dest is blackholed; can be dropped */ + BPF_FIB_LKUP_RET_UNREACHABLE, /* dest is unreachable; can be dropped */ + BPF_FIB_LKUP_RET_PROHIBIT, /* dest not allowed; can be dropped */ + BPF_FIB_LKUP_RET_NOT_FWDED, /* packet is not forwarded */ + BPF_FIB_LKUP_RET_FWD_DISABLED, /* fwding is not enabled on ingress */ + BPF_FIB_LKUP_RET_UNSUPP_LWT, /* fwd requires encapsulation */ + BPF_FIB_LKUP_RET_NO_NEIGH, /* no neighbor entry for nh */ + BPF_FIB_LKUP_RET_FRAG_NEEDED, /* fragmentation required to fwd */ +}; ``` > * BPF_FIB_LKUP_RET_NO_NEIGH: * Even if route lookup was a success, then the MAC-addresses are also * needed. This is obtained from arp/neighbour table, but if table is * (still) empty then BPF_FIB_LKUP_RET_NO_NEIGH is returned. To avoid * doing ARP lookup directly from XDP, then send packet to normal * network stack via XDP_PASS and expect it will do ARP resolution. ```go struct bpf_fib_lookup fib_params = {}; fib_params.family = AF_INET; fib_params.tos = ip4->tos; fib_params.l4_protocol = ip4->protocol; fib_params.sport = 0; fib_params.dport = 0; fib_params.tot_len = bpf_ntohs(ip4->tot_len); fib_params.ipv4_src = ip4->saddr; fib_params.ipv4_dst = ip4->daddr; fib_params.ifindex = ctx->ingress_ifindex; int rc = bpf_fib_lookup(ctx, &fib_params, sizeof(fib_params), BPF_FIB_LOOKUP_OUTPUT); ```

UPD:

          nr-gnb-159150  [007] d.s11  8770.527019: bpf_trace_printk: upf_ip_entrypoint start
          nr-gnb-159150  [007] d.s11  8770.527036: bpf_trace_printk: upf: gtp-u received
          nr-gnb-159150  [007] d.s11  8770.527041: bpf_trace_printk: upf: gtp pdu [ 10.100.50.236 -> 10.100.50.233 ]
          nr-gnb-159150  [007] d.s11  8770.527044: bpf_trace_printk: upf: uplink session for teid:1 far:1 headrm:0
          nr-gnb-159150  [007] d.s11  8770.527046: bpf_trace_printk: upf: far for teid:1 far:1 action:2
          nr-gnb-159150  [007] d.s11  8770.527054: bpf_trace_printk: upf: bpf_fib_lookup 4: 10.1.0.1 -> 172.17.0.1
          nr-gnb-159150  [007] d.s11  8770.527056: bpf_trace_printk: upf: bpf_fib_lookup nexthop: 172.17.0.1
     ksoftirqd/7-58      [007] d.s.1  8775.745280: bpf_trace_printk: upf_ip_entrypoint start
     ksoftirqd/7-58      [007] d.s.1  8775.745300: bpf_trace_printk: upf: arp received. passing to kernel
pirog-spb commented 1 year ago

As a fix now up calls fib_lookup without any flags. Previously BPF_FIB_LOOKUP_OUTPUT was used. So at the moment lookup is performed as well as for ingress side.