edgecomllc / eupf

5G User Plane Function (UPF) based on eBPF
Apache License 2.0
95 stars 14 forks source link

bpf_trace_printk: upf: no downlink session #514

Closed kouamdo closed 6 months ago

kouamdo commented 6 months ago

Hi , i'm trying to ping outside the NAT. But i'm facing one of many issue. There is no possibility for upf to forward packet to gNodeB:

image

since i'm trying to have every possibilities to debug or trace some traffic because since now , i'm not able to ping outside the NAT there is my docker compose file :

version: '2.4'

services:
  eupf:
    image: local/eupf:latest
    entrypoint: /bin/sh -c "ip route del default;ip route add default via 172.16.0.13 dev eth2 && sh /app/bin/entrypoint.sh --config /app/bin/eupf_config.yml"

    privileged: true
    environment:
      - GIN_MODE=release

    volumes:
      - /sys/fs/bpf:/sys/fs/bpf
      - /sys/kernel/debug:/sys/kernel/debug:ro
      - ./eupf_config.yml:/app/bin/eupf_config.yml
    ulimits:
      memlock: -1
    cap_add:
      - NET_ADMIN
      - SYS_ADMIN
    ports:
      - 2152:2152/udp
      - 8805:8805/udp
      - 8081:8081
      - 9091:9091
    restart: unless-stopped
    networks:
      n3:
        ipv4_address: 172.18.0.12
      n4:
        ipv4_address: 172.19.0.12
      n6:
        ipv4_address: 172.16.0.12
    sysctls:
      - net.ipv4.conf.all.forwarding=1

  net-tools:
    image: praqma/network-multitool:alpine-extra@sha256:47b259d4463950f5c10d9c0bf63d9e71ec456618f5549a414afa0c04392e0ac1
    privileged: true
    restart: unless-stopped
    networks:
      n6:
        ipv4_address: 172.16.0.13
    command:
      - /bin/sh
      - -c
      - |
        ip ro add 10.33.0.0/16 via 172.16.0.12 dev eth0
        echo "done"
        tail -f /dev/null

networks:
  n3:
    external: true
  n4:
    external: true
  n6:
    external: true

and this one is the config :

interface_name: [eth0,eth2]
xdp_attach_mode: generic
api_address: :8081
pfcp_address: :8805
pfcp_node_id: 172.19.0.12
metrics_address: :9091
n3_address:  172.18.0.12
feature_ftup: true
teid_pool: 65536

please tell me where i have mistaken.also , why the eupf can not send back packet to gnodeb ?

pirog-spb commented 6 months ago

Hi @kouamdo

According to debug log I would say that it's an routing/natting issue. There are no downlink packets in log.

I propose to capture pcap dump on host. Something like tcpdump -i any host <server ip>

kouamdo commented 6 months ago

the issues was inside the DN . I think it waas about the NAT. so i have make my own nat-dn container :

version: '2.4'

services:
  eupf:
    image: local/eupf:latest
    entrypoint:
      - /bin/sh
      - -c
      - |
        echo 1 > /proc/sys/net/ipv4/ip_forward
        ip route del default;
        ip route add default via 172.16.0.13 dev eth2 &&
        sh /app/bin/entrypoint.sh --config /app/bin/eupf_config.yml

    privileged: true
    environment:
      - GIN_MODE=release

    volumes:
      - /sys/fs/bpf:/sys/fs/bpf
      - /sys/kernel/debug:/sys/kernel/debug:ro
      - ./eupf_config.yml:/app/bin/eupf_config.yml
    ulimits:
      memlock: -1
    cap_add:
      - NET_ADMIN
      - SYS_ADMIN
    ports:
      - 2152:2152/udp
      - 8805:8805/udp
      - 8081:8081
      - 9091:9091
    restart: unless-stopped
    networks:
      n3:
        ipv4_address: 172.18.0.12
      n4:
        ipv4_address: 172.19.0.12
      n6:
        ipv4_address: 172.16.0.12
    sysctls:
      - net.ipv4.conf.all.forwarding=1

  nat-dn:
    image: ubuntu:focal
    privileged: true
    restart: unless-stopped
    networks:
      n6:
        ipv4_address: 172.16.0.13
    command:
      - /bin/sh
      - -c
      - |
        apt update && apt install -y iproute2 tcpdump iptables
        sysctl -w net.ipv6.conf.all.disable_ipv6=1
        echo 1 > /proc/sys/net/ipv4/ip_forward
        iptables -t nat -A POSTROUTING -s 10.33.0.0/16 -j MASQUERADE
        ip ro add 10.33.0.0/16 via 172.16.0.12 dev eth0
        echo "done"
        tail -f /dev/null

networks:
  n3:
    external: true
  n4:
    external: true
  n6:
    external: true

that's the output :

image

and that one is the tcpdump :

image