OpenVisualCloud / Media-Transport-Library

A real-time media transport(DPDK, AF_XDP, RDMA) stack for both raw and compressed video based on COTS hardware.
BSD 3-Clause "New" or "Revised" License
155 stars 47 forks source link

Native XDP on Intel X710 #891

Closed eugenevt closed 1 month ago

eugenevt commented 1 month ago

Cannot get it working, Ubuntu 22.04, kernel 6.5.0-35. I use maint-24.06 branch. I see that IGMP join is done, data flowing to the card, but nothing received. In MtlManager output I see following

libbpf: elf: skipping unrecognized data section(7) .xdp_run_config 
libbpf: elf: skipping unrecognized data section(7) xdp_metadata
libbpf: elf: skipping unrecognized data section(7) xdp_metadata

and following

[2024-06-04 18:00:54] [INFO] [Interface 3] Successfully inserted flow rule 7679 with queue 1
[2024-06-04 18:00:54] [ERROR] [Interface 3] Failed to update udp4_dp_filter map, dst_port: 16388

What I am missing?

frankdjx commented 1 month ago

For XDP, E810 is the one we only tested...

@ricmli Any input here?

eugenevt commented 1 month ago

The same on E810. Looks like my setup is not correct but I do not know what is wrong...

frankdjx commented 1 month ago

Not know why the ebpf filer map update is fail.

Failed to update udp4_dp_filter map, dst_port: 16388

A quick hack is allow all udp packet to XDP socket.

Change this line: https://github.com/OpenVisualCloud/Media-Transport-Library/blob/main/manager/mtl.xdp.c#L60 to

if (lookup_udp4_dp(dst_port) == 0) return XDP_DROP;
eugenevt commented 1 month ago

Recompiled MTL with gcc (was clang 14), now I have no error, but still no data. MTL kernel on the same interface works fine.

eugenevt commented 1 month ago

After recompiling xdp-tools using GCC, I was able to get the data on the X710 board, do not know what is wrong with my E810-QDA2 card, but installed in the same PC same slot it does not allow me do SRIOV, so maybe it is somehow affects XDP as well. So using GCC is critical, needs to be pointed in the documentation.

frankdjx commented 1 month ago

We are using GCC most time, not understand there's any special reason to stop using Clang for XDP.

For E810, do you still see update udp4_dp_filter map fail? SRIOV is not required for XDP per my understanding, and can you share both MtlManager and IMTL running log?

ricmli commented 1 month ago

Looks like some xdp-tools/libbpf issue with clang.

eugenevt commented 1 month ago

I can confirm that MtlManager compiled with clang, gives map update error, compiled with gcc not, on both cards.

frankdjx commented 1 month ago

https://github.com/OpenVisualCloud/Media-Transport-Library/pull/896 should fix the update error, at least it update successfully in my local clang build.

eugenevt commented 1 month ago

I have tested the fix, can confirm that now clang compilation works, I have the data.