edgecomllc / eupf

5G User Plane Function (UPF) based on eBPF
Apache License 2.0
106 stars 20 forks source link

OpenAirInterface project research #340

Closed pirog-spb closed 10 months ago

pirog-spb commented 1 year ago

Now OpenAirInterface supports radio emulation. Let's verify compatibility with eUPF.

https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/doc/RUNMODEM.md#l2-nfapi-simulator

Acceptance criteria:

PapaySail commented 1 year ago

L2 nfapi simulator https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/doc/L2NFAPI.md

OAI Full Stack 5G-NR L2 simulation with containers and a proxy. https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/ci-scripts/yaml_files/5g_l2sim_tdd/README.md

This nFAPI feature and the proxy are still under development. At time of writing, we were able to run in host-mode, 1 NR-UE and just ping traffic.

PapaySail commented 1 year ago

deploy the whole project

Details

```ruby sergo@edgecom:~/gitlab$ git clone https://gitlab.eurecom.fr/oai/openairinterface5g.git sergo@edgecom:~/gitlab$ cd openairinterface5g/ci-scripts/yaml_files/5g_l2sim_tdd/ nano docker-compose.yaml - DEFAULT_DNS_IPV4_ADDRESS=169.254.25.10 #172.21.3.100 nano ../../conf_files/gnb.sa.band78.106prb.l2sim.conf NETWORK_INTERFACES : { GNB_INTERFACE_NAME_FOR_NG_AMF = "ens3"; GNB_IPV4_ADDRESS_FOR_NG_AMF = "188.120.232.247"; GNB_INTERFACE_NAME_FOR_NGU = "ens3"; GNB_IPV4_ADDRESS_FOR_NGU = "188.120.232.247"; GNB_PORT_FOR_NGU = 2152; # Spec 2152 }; nano ../../conf_files/nrue.band78.106prb.l2sim.conf MACRLCs = ( { num_cc = 1; tr_n_preference = "nfapi"; local_n_if_name = "ens3"; remote_n_address = "127.0.0.1"; //Proxy IP local_n_address = "127.0.0.1"; sudo docker pull oaisoftwarealliance/proxy:develop sudo docker image tag oaisoftwarealliance/proxy:develop oai-lte-multi-ue-proxy:latest sudo ifconfig lo: 127.0.0.2 netmask 255.0.0.0 up sudo docker-compose up -d mysql oai-nrf oai-amf oai-smf oai-spgwu oai-ext-dn sergo@edgecom:~$ ip a | grep "l2sim" 6882: l2sim-public: mtu 1500 qdisc noqueue state UP group default inet 192.168.71.129/26 brd 192.168.71.191 scope global l2sim-public 6883: l2sim-traffic: mtu 1500 qdisc noqueue state UP group default inet 192.168.72.129/26 brd 192.168.72.191 scope global l2sim-traffic 6885: veth40c5704@if6884: mtu 1500 qdisc noqueue master l2sim-public state UP group default 6887: veth1f6fe02@if6886: mtu 1500 qdisc noqueue master l2sim-public state UP group default 6889: vethfeef02a@if6888: mtu 1500 qdisc noqueue master l2sim-public state UP group default 6891: vethed74c67@if6890: mtu 1500 qdisc noqueue master l2sim-public state UP group default 6893: vetha74c625@if6892: mtu 1500 qdisc noqueue master l2sim-public state UP group default 6895: veth3e62ba9@if6894: mtu 1500 qdisc noqueue master l2sim-traffic state UP group default 6897: veth31a884b@if6896: mtu 1500 qdisc noqueue master l2sim-traffic state UP group default sudo docker-compose up -d oai-gnb ``` `gNB` is connected with the `AMF`: ```prolog sudo docker logs l2sim-oai-amf .... [2023-10-31T06:56:36.746510] [AMF] [amf_n2 ] [debug] Sending NG_SETUP_RESPONSE Ok [2023-10-31T06:56:36.746513] [AMF] [amf_n2 ] [debug] gNB with gNB_id 0xe000, assoc_id 614 has been attached to AMF [2023-10-31T06:56:36.746525] [AMF] [ngap ] [debug] Free NGAP Message PDU [2023-10-31T06:56:36.746538] [AMF] [ngap ] [debug] Free NGAP Message PDU [2023-10-31T06:56:47.352636] [AMF] [amf_app] [info ] [2023-10-31T06:56:47.352674] [AMF] [amf_app] [info ] |----------------------------------------------------------------------------------------------------------------| [2023-10-31T06:56:47.352676] [AMF] [amf_app] [info ] |----------------------------------------------------gNBs' information-------------------------------------------| [2023-10-31T06:56:47.352678] [AMF] [amf_app] [info ] | Index | Status | Global ID | gNB Name | PLMN | [2023-10-31T06:56:47.352686] [AMF] [amf_app] [info ] | 1 | Connected | 0xe000 | gnb-l2sim-vnf | 208, 99 | [2023-10-31T06:56:47.352689] [AMF] [amf_app] [info ] |----------------------------------------------------------------------------------------------------------------| ....... ``` Deploy OAI NR-UE**0** and proxy ```ruby sudo docker-compose up -d proxy oai-nr-ue0 ``` ```ryby sergo@edgecom:~/gitlab/openairinterface5g/ci-scripts/yaml_files/5g_l2sim_tdd$ sudo docker stats --no-stream --format "table {{.Container}}\t{{.CPUPerc}} {{.MemUsage}}\t{{.MemPerc}}" l2sim-mysql l2sim-oai-amf l2sim-oai-ext-dn l2sim-oai-gnb l2sim-oai-nr-ue0 l2sim-oai-nrf l2sim-oai-smf l2sim-oai-spgwu l2sim-proxy CONTAINER CPU % MEM USAGE / LIMIT MEM % l2sim-mysql 0.79% 377.4MiB / 11.69GiB 3.15% l2sim-oai-amf 0.00% 11.26MiB / 11.69GiB 0.09% l2sim-oai-ext-dn 0.00% 4.582MiB / 11.69GiB 0.04% l2sim-oai-gnb 4.41% 159.3MiB / 11.69GiB 1.33% l2sim-oai-nr-ue0 0.00% 0B / 0B 0.00% l2sim-oai-nrf 1.27% 8.449MiB / 11.69GiB 0.07% l2sim-oai-smf 0.00% 9.988MiB / 11.69GiB 0.08% l2sim-oai-spgwu 4.20% 8.977MiB / 11.69GiB 0.08% l2sim-proxy 20.45% 18.11MiB / 11.69GiB 0.15% ```

📝 CPU speed matters from https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/develop/ci-scripts/yaml_files/5g_l2sim_tdd/README.md

But the CPU speed matters; I am running on a fast server:

...
Model name:          Intel(R) Xeon(R) Silver 4215R CPU @ 3.20GHz

I tried on a slower server with more CPUs and it did not work:

...
Model name:          Intel(R) Xeon(R) CPU E5-2658 v3 @ 2.20GHz

Our CPU is

sergo@edgecom:~$ lscpu
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         46 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  8
  On-line CPU(s) list:   0-7
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz
PapaySail commented 1 year ago

✔ oai-nr-ue0 successfully connected

Details

```ruby root@edgecom:/opt/oai-nr-ue# ip a |grep oai 6934: oaitun_ue1: mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 500 inet 12.1.1.3/24 brd 12.1.1.255 scope global oaitun_ue1 root@edgecom:/opt/oai-nr-ue# ping -I oaitun_ue1 -c 20 192.168.72.135 PING 192.168.72.135 (192.168.72.135) from 12.1.1.3 oaitun_ue1: 56(84) bytes of data. 64 bytes from 192.168.72.135: icmp_seq=1 ttl=63 time=192 ms 64 bytes from 192.168.72.135: icmp_seq=2 ttl=63 time=135 ms 64 bytes from 192.168.72.135: icmp_seq=3 ttl=63 time=235 ms ^C --- 192.168.72.135 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2063ms rtt min/avg/max/mdev = 135.479/187.748/235.318/40.894 ms root@edgecom:/opt/oai-nr-ue# ping -I oaitun_ue1 -c 3 1.1.1.1 PING 1.1.1.1 (1.1.1.1) from 12.1.1.3 oaitun_ue1: 56(84) bytes of data. 64 bytes from 1.1.1.1: icmp_seq=1 ttl=54 time=1155 ms 64 bytes from 1.1.1.1: icmp_seq=2 ttl=54 time=390 ms 64 bytes from 1.1.1.1: icmp_seq=3 ttl=54 time=104 ms --- 1.1.1.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2006ms rtt min/avg/max/mdev = 104.483/549.893/1155.238/443.625 ms, pipe 2 root@edgecom:/opt/oai-nr-ue# apt-get update root@edgecom:/opt/oai-nr-ue# apt-get install traceroute root@edgecom:/opt/oai-nr-ue# traceroute -ni oaitun_ue1 1.1.1.1 traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets 1 12.1.1.1 193.342 ms 204.240 ms 207.290 ms 2 * * * 3 10.0.0.1 209.333 ms 209.291 ms 209.252 ms 4 172.31.141.1 214.508 ms 214.516 ms 214.497 ms 5 172.17.23.111 209.126 ms 209.107 ms 209.086 ms 6 80.239.193.150 211.423 ms 80.64.102.134 219.827 ms 80.239.193.150 220.453 ms 7 80.64.97.75 217.485 ms 215.339 ms 80.239.193.241 217.142 ms 8 1.1.1.1 215.201 ms 215.202 ms 80.64.97.74 217.050 ms ```

sergo@edgecom:~/gitlab/openairinterface5g/ci-scripts/yaml_files/5g_l2sim_tdd$ sudo docker stats --no-stream --format "table {{.Container}}\t{{.CPUPerc}}
       {{.MemUsage}}\t{{.MemPerc}}" l2sim-mysql l2sim-oai-amf l2sim-oai-ext-dn l2sim-oai-gnb l2sim-oai-nr-ue0 l2sim-oai-nrf l2sim-oai-smf l2sim-oai-spgwu l2sim-proxy
CONTAINER          CPU %     MEM USAGE / LIMIT     MEM %
l2sim-mysql        0.63%     377.8MiB / 11.69GiB   3.16%
l2sim-oai-amf      6.22%     11.56MiB / 11.69GiB   0.10%
l2sim-oai-ext-dn   0.00%     6.02MiB / 11.69GiB    0.05%
l2sim-oai-gnb      5.75%     691.3MiB / 11.69GiB   5.78%
l2sim-oai-nr-ue0   0.00%     0B / 0B               0.00%
l2sim-oai-nrf      1.15%     7.672MiB / 11.69GiB   0.06%
l2sim-oai-smf      0.01%     33.62MiB / 11.69GiB   0.28%
l2sim-oai-spgwu    0.00%     12.25MiB / 11.69GiB   0.10%
l2sim-proxy        26.29%    1.045GiB / 11.69GiB   8.94%

There is picture of mostly the same demo architecture at https://gitlab.eurecom.fr/oai/openairinterface5g/-/tree/develop/ci-scripts/yaml_files/5g_rfsimulator#2-deploy-containers

Deployment