Xilinx-CNS / onload

OpenOnload high performance user-level network stack
Other
545 stars 90 forks source link

ubuntu20.04.2LTS ,ixgbe 5.1.0-K, not working with EF_AF_XDP_ZEROCOPY=1 #18

Open density1970 opened 3 years ago

density1970 commented 3 years ago

Hello , I have been testing Onload with AF_XDP support on 10 GBe Intel 82599. my Environment Details: CPU: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) OS: Ubuntu 20.04.2 LTS Kernel: 5.4.0-65-generic

root@xcsc:# ethtool -i enp4s0f1 driver: ixgbe version: 5.1.0-k firmware-version: 0x000161ae expansion-rom-version: bus-info: 0000:04:00.1 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes

before I enable the EF_AF_XDP_ZEROCOPY to 1 in the latency.opf, my application(onload -p ./conf/latency.opf ./zxTrade) works fine:

Recv tickData(15:29:05): { MkDataCounts: 23840 , TradeDataCounts: 0 , OrderDataCounts: 0 } Recv tickData(15:30:05): { MkDataCounts: 26224 , TradeDataCounts: 0 , OrderDataCounts: 0 }

after I enable the EF_AF_XDP_ZEROCOPY to 1 in the latency.opf:

onload_set EF_AF_XDP_ZEROCOPY 1

stack can be created: root@xcsc:# onload_stackdump stacks

stack-id stack-name pids

1 - 9499

EF_AF_XDP_ZEROCOPY enabled: root@xcsc:# onload_stackdump lots | grep XDP EF_XDP_MODE: 0 EF_AF_XDP_ZEROCOPY: 1 (default: 0) env: EF_AF_XDP_ZEROCOPY=1

but my application(onload -p ./conf/latency.opf ./zxTrade) can not work fine,application show no data packets received;

Recv tickData: { MkDataCounts:0, TradeDataCounts:0, OrderDataCounts:0 } Recv tickData: { MkDataCounts:0, TradeDataCounts:0, OrderDataCounts:0 }

the thing below drived me crazy: when I delete "the onload_set EF_AF_XDP_ZEROCOPY 1" in latency.opf, and reload my application, the system crashed, need to reboot. After the Ubuntu system reboot, dmesg show these message: root@xcsc:~# dmesg | grep onload [ 14.229749] [onload] Onload [ 14.229750] [onload] Copyright 2019-present Xilinx, 2006-2019 Solarflare Communications, 2002-2005 Level 5 Networks [ 14.489442] onload_cp_server[461]: Spawned daemon process 481 [ 14.813890] synth uevent: /devices/virtual/onload/onload: failed to send uevent [ 14.813893] onload onload: uevent: failed to send synthetic uevent [ 14.832521] synth uevent: /devices/virtual/onload_epoll/onload_epoll: failed to send uevent [ 14.832523] onload_epoll onload_epoll: uevent: failed to send synthetic uevent

Is ixgbe driver 5.1.0-K(come with Ubuntu20.04.2 LTS) is not support AF_XDP, ZEROCOPY or other reason ???

thanks.

ol-alexandra commented 3 years ago

@maciejj-xilinx & I have agreed that it is probably ON-12643/SWNETLINUX-3906 internal bugs. I'm working on it.