Data is not working with UPG-VPP integrated with Open5GS

[BG00818733]

Hi,

I have setup my 5G Core with Open5GS integrated with UPG-VPP (UPF) on AWS infrastructure as per installation guides. https://github.com/s5uishida/open5gs_5gc_ueransim_vpp_upf_dpdk_sample_config https://github.com/s5uishida/install_vpp_upf_dpdk

MY UE is getting registered, but data is not working.

my upg configurations are. startup.conf unix { nodaemon log /tmp/vpp.log full-coredump gid vpp interactive cli-listen /run/vpp/cli.sock exec /root/openair-upf/init.conf }

api-trace { on }

cpu { main-core 0 corelist-workers 1-3 }

api-segment { gid vpp }

dpdk { dev 0000:00:06.0 {name n3} dev 0000:00:07.0 {name n4} dev 0000:00:08.0 {name n6} }

plugins { path /usr/lib/x86_64-linux-gnu/vpp_plugins/ plugin ikev2_plugin.so {disable} plugin dpdk_plugin.so {enable} plugin upf_plugin.so {enable} }

init.conf set interface ip table n6 0 set interface mtu 9000 n6 set interface ip address n6 10.1.12.6/28 set interface state n6 up

set interface ip table n4 0 set interface mtu 9000 n4 set interface ip address n4 10.1.11.6/28 set interface state n4 up

set interface ip table n3 0 set interface mtu 9000 n3 set interface ip address n3 10.1.10.6/28 set interface state n3 up

ip route add 0.0.0.0/0 table 0 via 10.1.12.14 n6

upf pfcp endpoint ip 10.1.11.6 vrf 0

upf node-id fqdn 10.1.11.6

upf nwi name internet vrf 0

upf specification release 16

upf gtpu endpoint ip 10.1.12.6 nwi internet teid 0x000004d2/2

[BG00818733]

One additional thing while starting the UPG VPP, i am getting below error.

root@ip-10-1-1-180:/home/ubuntu# /usr/bin/vpp -c /root/openair-upf/startup.conf dpdk [warn ]: not enough DPDK crypto resources dpdk [warn ]: unsupported tx offloads requested on port 0: multi-segs dpdk [warn ]: unsupported tx offloads requested on port 1: multi-segs dpdk [warn ]: unsupported tx offloads requested on port 2: multi-segs dpdk/cryptodev [warn ]: dpdk_cryptodev_init: Not enough cryptodevs

---

/ / \ () | | / / \/ \ / // // / / / \ | |/ / / / // /__()/___/ |__// /_/

vpp# interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 0: upip_ip_interface_ip:1298: No NWI IP found, using first interface IP 0: upip_ip_interface_ip:1298: No NWI IP found, using first interface IP

Please suggest your inputs and help us to fix this.

[BG00818733]

UPF Session output:

vpp# show upf session CP F-SEID: 0x00000000000005a9 (1449) @ 10.1.11.11 UP F-SEID: 0x00000000000005a9 (1449) @ 10.1.11.6 PFCP Association: 0 TEID assignment per choose ID PDR: 1 @ 0x7f1dc72bf2b0 Precedence: 255 PDI: Fields: 0000000c Source Interface: Core Network Instance: internet UE IP address (destination): IPv4 address: 10.45.0.2 SDF Filter [1]: permit out ip from any to assigned Outer Header Removal: no FAR Id: 1 URR Ids: [1] @ 0x7f1dc6432f30 QER Ids: [1] @ 0x7f1dc6432f60 PDR: 2 @ 0x7f1dc72bf330 Precedence: 255 PDI: Fields: 0000000d Source Interface: Access Network Instance: internet Local F-TEID: 827107826 (0x314ca9f2) IPv4: 10.1.12.6 UE IP address (source): IPv4 address: 10.45.0.2 SDF Filter [1]: permit out ip from any to assigned Outer Header Removal: GTP-U/UDP/IPv4 FAR Id: 2 URR Ids: [] @ 0x0 QER Ids: [1] @ 0x7f1dc72c0580 PDR: 3 @ 0x7f1dc72bf3b0 Precedence: 1000 PDI: Fields: 00000001 Source Interface: CP-function Network Instance: internet Local F-TEID: 564920489 (0x21ac00a9) IPv4: 10.1.12.6 Outer Header Removal: GTP-U/UDP/IPv4 FAR Id: 1 URR Ids: [] @ 0x0 QER Ids: [1] @ 0x7f1dc72c05b0 PDR: 4 @ 0x7f1dc72bf430 Precedence: 1 PDI: Fields: 00000009 Source Interface: Access Network Instance: internet Local F-TEID: 827107826 (0x314ca9f2) IPv4: 10.1.12.6 SDF Filter [1]: permit out 58 from ff02::2 to assigned Outer Header Removal: GTP-U/UDP/IPv4 FAR Id: 3 URR Ids: [] @ 0x0 QER Ids: [] @ 0x0 FAR: 1 Apply Action: 00000002 == [FORWARD] Forward: Network Instance: internet Destination Interface: 0 Outer Header Creation: [GTP-U/UDP/IPv4],TEID:00000001,IP:10.1.10.11 FAR: 2 Apply Action: 00000002 == [FORWARD] Forward: Network Instance: internet Destination Interface: 1 FAR: 3 Apply Action: 00000002 == [FORWARD] Forward: Network Instance: internet Destination Interface: 3 Outer Header Creation: [GTP-U/UDP/IPv4],TEID:00000001,IP:10.1.11.11 URR: 1 Measurement Method: 0002 == [VOLUME] Reporting Triggers: 0002 == [VOLUME THRESHOLD] Status: 0 == [] Start Time: 2024/02/27 11:29:23:214 vTime of First Usage: 0.0000 vTime of Last Usage: 0.0000 Volume Up: Measured: 0, Theshold: 0, Pkts: 0 Consumed: 0, Quota: 0 Down: Measured: 0, Theshold: 0, Pkts: 0 Consumed: 0, Quota: 0 Total: Measured: 0, Theshold: 104857600, Pkts: 0 Consumed: 0, Quota: 0

[s5uishida]

@BG00818733

I haven't tried it, but the following may be helpful.

https://s3-docs.fd.io/vpp/24.02/usecases/vppcloud/vppinaws.html

$ sudo modprobe uio
$ sudo insmod kmod/igb_uio.ko

Are you binding your network devices with this module?

[BG00818733]

Hi @s5uishida. Thanks for your response. Currently we are binding the ENA devices with vfio-pci

Please suggest your inputs

[s5uishida]

@BG00818733

You may try igb_uio instead of vfio-pci. example)

dpdk-devbind.py -b igb_uio 0000:00:06.0 --force

[BG00818733]

@s5uishida Will it work for Ubuntu 22.04 with EC2 Instance Type c5.2xl.

Actually I am getting below error while loading igb_uio. modprobe igb_uio modprobe: FATAL: Module igb_uio not found in directory /lib/modules/6.2.0-1017-aws

Please suggest

[BG00818733]

Hi @s5uishida With Ubuntu 20.04. I have installed the igb_uio using the command. sudo apt-get install -y dpdk-igb-uio-dkms

Now After binding the ENA with igb_ui, still we are facing the same issues. dpdk-devbind.py -b igb_uio 0000:00:06.0 --force dpdk-devbind.py -b igb_uio 0000:00:07.0 --force dpdk-devbind.py -b igb_uio 0000:00:08.0 --force

Please suggest your inputs.

[BG00818733]

Hi @s5uishida,

Can below errors could be the reason for data not working. Please share your views on this.

vpp# interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 0: upip_ip_interface_ip:1298: No NWI IP found, using first interface IP 0: upip_ip_interface_ip:1298: No NWI IP found, using first interface IP

[BG00818733]

Hi @s5uishida

We tried further with m5 instance type categories on AWS EC2 and facing the same dpdk binding errors as mentioned above. Additionally, we tried the same in VMs on a baremetal server and we are not seeing any errors.

Hence, we request your support for the difference in Cloud Infrastructure which is causing below dpdk binding errors.

root@ip-10-1-1-244:/home/ubuntu# /usr/bin/vpp -c /root/openair-upf/startup.conf clib_sysfs_prealloc_hugepages:260: pre-allocating 20 additional 2048K hugepages on numa node 0 0: clib_sysfs_prealloc_hugepages:260: pre-allocating 8 additional 2048K hugepages on numa node 0 dpdk [warn ]: not enough DPDK crypto resources dpdk [warn ]: unsupported tx offloads requested on port 0: multi-segs dpdk [warn ]: unsupported tx offloads requested on port 1: multi-segs dpdk [warn ]: unsupported tx offloads requested on port 2: multi-segs dpdk/cryptodev [warn ]: dpdk_cryptodev_init: Not enough cryptodevs

---

/ / \ () | | / / \/ \ / // // / / / \ | |/ / / / // /__()/___/ |__// /_/

vpp# interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95 interface [error ]: hw_add_del_mac_address: dpdk_add_del_mac_address: mac address add/del failed: -95

Your inputs will be appreciated.

Thanks and Regards, Bharat Gupta

[BG00818733]

Hi All,

This issues worked for us by disabling the source/destination check in AWS. Hence closing this issue.

Regards, Bharat