search

edgecomllc / eupf

5G User Plane Function (UPF) based on eBPF
Apache License 2.0
95 stars 14 forks source link

feat: Support Traffic Steering for Service Function Chaining #549

Open tariromukute opened 1 month ago

tariromukute commented 1 month ago

Firstly, thank you for this project!

This pull request introduces support for Traffic Steering within the Service Function Chaining (SFC) framework. This enables the User Plane Function (UPF) to:

The traffic steering implementation enables the N6-LAN for perform Service Function Chaining on the traffic after it leaves the UPF but before the internet. Checkout the OpenN6LAN project.

Key Changes:

  1. Modifications to the UPF to handle forwarding policies in the FAR.
  2. Implementation of NSH encapsulation logic within the UPF based on the presence of forwarding policy identifiers.
  3. Updates to handle NSH headers during packet processing

Testing:

The changes can be tested by setting up the eUPF and installing FAR rules with forwarding policy identifiers. Please see below. The setup is depicted below, along with the files for setting up the environment for test.

representation_diagram

  1. Sessions rules to install on eUPF using pfcp-kitchen-sink, sessions-eupf.yaml in the docker compose below

    - seid: 0
pdrs:
- pdrID: 0
precedence: 0
pdi:
  sourceInterface: Access
  networkInstance: access.oai.org
  localFTEID:
    teid: 1234
    ip4: 192.168.71.134
  ueIPAddress:
    isDestination: false
    ip4: 12.1.1.2
outerHeaderRemoval: OUTER_HEADER_GTPU_UDP_IPV4
farID: 12
- pdrID: 1
precedence: 0
pdi:
  sourceInterface: SGiLAN
  networkInstance: core.oai.org
  ueIPAddress:
    isDestination: true
    ip4: 12.1.1.2
farID: 13
fars:
- farID: 12
applyAction: Forward
forwardingParameters:
  destinationInterface: SGiLAN
  networkInstance: core.oai.org
- farID: 13
applyAction: Forward
forwardingParameters:
  destinationInterface: Access
  networkInstance: access.oai.org
  outerHeaderCreation:
    desc: OUTER_HEADER_CREATION_GTPU_UDP_IPV4
    teid: 1234
    ip: 192.168.71.130
- seid: 1
pdrs:
- pdrID: 2
precedence: 0
pdi:
  sourceInterface: Access
  networkInstance: access.oai.org
  localFTEID:
    teid: 1235
    ip4: 192.168.71.134
  ueIPAddress:
    isDestination: false
    ip4: 12.1.1.3
outerHeaderRemoval: OUTER_HEADER_GTPU_UDP_IPV4
farID: 14
- pdrID: 3
precedence: 0
pdi:
  sourceInterface: SGiLAN
  networkInstance: core.oai.org
  ueIPAddress:
    isDestination: true
    ip4: 12.1.1.3
farID: 15
fars:
- farID: 14
applyAction: Forward
forwardingParameters:
  destinationInterface: SGiLAN
  networkInstance: core.oai.org
  forwardingPolicy: "MTEyMjg3OQ=="
- farID: 15
applyAction: Forward
forwardingParameters:
  destinationInterface: Access
  networkInstance: access.oai.org
  outerHeaderCreation:
    desc: OUTER_HEADER_CREATION_GTPU_UDP_IPV4
    teid: 1235
    ip: 192.168.71.130

  2. Docker compose for setting up components for testing

    
version: '3.8'
services:
pfcp-kitchen-sink:
    container_name: "pfcp-kitchen-sink"
    image: tariromukute/pfcp-kitchen-sink:latest
    volumes:
        - ./sessions-eupf.yaml:/app/sessions.yaml
    command: ./pfcpclient -r 192.168.70.134:8805 -s sessions.yaml
    depends_on:
        - edgecomllc-eupf
    networks:
        n4_net:
            ipv4_address: 192.168.70.131
ue-sim:
    privileged: true # So it can create UE namespaces
    container_name: "ue-sim"
    image: tariromukute/tc-gtpu:latest
    command:
    - /bin/bash
    - -c
    - |
        apt update -y;
        apt install curl -y;

        mkdir -p /etc/netns/uegtp0/
        echo 'nameserver 8.8.8.8' > /etc/netns/uegtp0/resolv.conf
        mkdir -p /etc/netns/uegtp1/
        echo 'nameserver 8.8.8.8' > /etc/netns/uegtp1/resolv.conf

        ./tc-gtpu -g eth0 -i uegtp -s 192.168.71.130 -d 192.168.71.134 \
        -u 12.1.1.2 -b 12.1.1.1 --ul-teid 1234 --dl-teid 1234 --qfi 9 \
        -n 2 -f /home/tu-gtpu.pcap -vvv
    # healthcheck:
    #     test: ip netns exec uegtp0 ping -c 4 192.168.73.129 || exit 1
    #     interval: 10s
    #     timeout: 5s
    #     retries: 5
    volumes:
        - /sys/kernel/debug/:/sys/kernel/debug/
        - /sys/fs/bpf:/sys/fs/bpf
    devices:
        - /dev/net/tun:/dev/net/tun
    depends_on:
        - edgecomllc-eupf
        - pfcp-kitchen-sink
    networks:
        n3_net:
            ipv4_address: 192.168.71.130

edgecomllc-eupf:
    platform: linux/amd64
    container_name: "edgecomllc-eupf"
    image: tariromukute/edgecomllc-eupf:sfc-latest
    entrypoint:
    - /bin/sh
    - -c
    - |
        sysctl -w net.ipv4.conf.eth2.send_redirects=0;
        sysctl -w net.ipv4.conf.all.send_redirects=0;
        ip route del default;
        ip route add default via 192.168.72.138 dev eth2 &&
        sh /app/bin/entrypoint.sh
    environment:
        - UPF_INTERFACE_NAME=eth0,eth2
        - UPF_XDP_ATTACH_MODE=generic
        - UPF_API_ADDRESS=:8080
        - UPF_PFCP_ADDRESS=:8805
        - UPF_METRICS_ADDRESS=:9091
        - UPF_PFCP_NODE_ID=192.168.70.134
        - UPF_N3_ADDRESS=192.168.71.134
        - UPF_UEIP_POOL=12.1.1.0/24
        - UPF_LOGGING_LEVEL=debug
    cap_add:
        - NET_ADMIN
        - SYS_ADMIN
        - SYS_RESOURCE # setrlimit
    cap_drop:
        - ALL
    ports:
        - "127.0.0.1:8081:8081"
        - "127.0.0.1:8880:8080"
        - "127.0.0.1:9090:9090"
    sysctls:
        - net.ipv4.conf.all.forwarding=1
    privileged: true
    networks:
        n4_net:
            ipv4_address: 192.168.70.134
        n3_net:
            ipv4_address: 192.168.71.134
        n6_net:
            ipv4_address: 192.168.72.134
            mac_address: 02:42:ac:11:65:43
n6-lan:
    platform: linux/amd64
    privileged: true
    init: true
    container_name: "n6-lan"
    image: tariromukute/openn6lan-ovs:latest
    command: 
    - /bin/bash
    - -c
    - |
        sh testovs.sh
        # ovs-vsctl add-br brovs1
        ovs-vsctl add-port brovs1 eth1
        ip addr flush dev eth1 && ip addr add 192.168.72.138/26 dev brovs1 && ip link set brovs1 up
        ip route add 12.1.1.0/24 via 192.168.72.134 dev brovs1

        ovs-vsctl add-port brovs1 eth0
        ip addr flush dev eth0 && ip addr add 192.168.73.138/26 dev brovs1 && ip link set brovs1 up
        iptables -t nat -A POSTROUTING -o brovs1 -j MASQUERADE

        ip route del default
        ip route add default via 192.168.73.129 dev brovs1

        ip link set dev eth1 xdpgeneric obj /app/nsh-decap.bpf.o sec xdp_nsh_decap

        # Without this setup packets are not forwarded
        sh /app/ovs/install-static-rules.sh

        sysctl -w net.ipv4.conf.all.send_redirects=0
        sysctl -w net.ipv4.conf.brovs1.send_redirects=0
        sysctl -w net.ipv4.conf.eth0.send_redirects=0
        sysctl -w net.ipv4.conf.eth1.send_redirects=0

        tail -f /dev/null
    devices:
        - /dev/net/tun:/dev/net/tun # https://docs.openvswitch.org/en/stable/intro/install/userspace/#building-and-installing
    volumes:
        - /lib/modules:/lib/modules
    deploy:
        resources:
            reservations:
                memory: 2G
    networks:
        n6_net:
            ipv4_address: 192.168.72.138
        data_net:
            ipv4_address: 192.168.73.138

iperf3:
    privileged: true
    platform: linux/amd64
    container_name: "iperf3"
    image: ubuntu:jammy
    command: 
    - /bin/bash
    - -c
    - |
        apt update -y
        apt install iperf3 -y
        iperf3 -s
    cap_add:
        - NET_ADMIN
    networks:
        data_net:
            ipv4_address: 192.168.73.137

networks: n4_net: driver: bridge name: demo-n4-net ipam: config:

  1. Send traffic using the emulated UE. Note: you have to start with the UE without a policy for NSH since the FIB tables need to have been populate prior.
# UE with ip 12.1.1.2
docker exec -it ue-sim \
    ip netns exec uegtp0 ping -c 4 8.8.8.8

# UE with ip 12.1.1.3    
docker exec -it ue-sim \
    ip netns exec uegtp1 ping -c 4 8.8.8.8
tariromukute commented 1 month ago

Thanks @pirog-spb for the comments, I have updated the code.