search

Xilinx-CNS / onload

OpenOnload high performance user-level network stack
Other
580 stars 95 forks source link

af_xdp solution not working on intel 82599 #10

Open sarosh-arif opened 3 years ago

sarosh-arif commented 3 years ago

I followed the following steps:

  1. cloned this repository
  2. used the steps in DEVELOPING.md to build from repository
  3. ran ./onload_tool reload
  4. wrote the name of the interface in /sys/module/sfc_resource/afxdp/register with sudo privileges
  5. ran my application using the following syntax: "sudo onload ./bts_sut" (it is a udp to tcp application. It receives udp packets from a certain interface and sends tcp replies)

On running this application I get the following output:

onload: WARNING: EF_TCP_SYNRECV_MAX=16384 and EF_MAX_ENDPOINTS=8192 are inconsistent.
onload: EF_TCP_SYNRECV_MAX is set to 16384 based on /proc/sys/net/ipv4/tcp_max_syn_backlog value and assuming up to 4 listening sockets in the Onload stack
onload: Too few endpoints requested: ~4 syn-receive states consume one endpoint. 
oo:bts_sut[13180]: Using Onload 0d46412079ceb6c162e6cb5ceabf26d610c5a5d4 [6]
oo:bts_sut[13180]: Copyright 2019-2021 Xilinx, 2006-2019 Solarflare Communications, 2002-2005 Level 5 Networks

TCP Connection Failed

When I check syslog I get the following information: 

Feb 10 14:28:34 bts-s1 kernel: [12726.892817] [onload] [6]: WARNING: huge pages are incompatible with AF_XDP. Disabling hugepage support.
Feb 10 14:28:34 bts-s1 kernel: [12726.916274] [onload] oof_socket_add_full_hw: 6:3 ERROR: FILTER TCP 10.0.0.17:57471 10.0.0.21:29999 failed (-95)
Feb 10 14:28:34 bts-s1 kernel: [12726.916966] [sfc efhw] af_xdp_flush_rx_dma_channel: FIXME AF_XDP
Feb 10 14:28:34 bts-s1 kernel: [12726.917166] [sfc efhw] af_xdp_flush_tx_dma_channel: FIXME AF_XDP

Machine specification are as follows::

  1. 82599 NIC
  2. Operating System: Ubuntu 18.04.4 LTS
  3. Kernel: Linux 5.7.0+
  4. Machine: Intel Xeon D2166

I use this application often, it works perfectly fine without onload af_xdp solution. I was looking into this in order to accelerate my application without changing the source code Is there anything that I am doing wrong? Or is there anything I can do to make this work?

maciejj-xilinx commented 3 years ago

Hi @sarosh-arif ,

82599 requres explicitely enabling n-tuples prior to creating onload stack ethtool --features enp4s0f0 ntuple

Hope that helps, Maciej

h2cw2l commented 3 years ago

Hello, is this problem solved ? I had the same problem.

Thanks!

h2cw2l commented 3 years ago

Hi @sarosh-arif ,

82599 requres explicitely enabling n-tuples prior to creating onload stack ethtool --features enp4s0f0 ntuple

Hope that helps, Maciej

Hello @maciejj-xilinx , What do you mean, I can not find this command.