Mellanox / libxlio

Other
42 stars 21 forks source link

After running the command in the cloud environment, the error keeps looping. #262

Open ty-dc opened 13 hours ago

ty-dc commented 13 hours ago

Subject

Issue type

Configuration:

XLIO INFO   : XLIO_VERSION: 3.40.2-1 Release built on Oct  7 2024 17:14:06 -*- DEBUG -*-

~# cat /etc/os-release PRETTY_NAME="Ubuntu 22.04.4 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.4 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=jammy


* OFED

~# ofed_info -s MLNX_OFED_LINUX-24.10-0.7.0.0:

* Hardware

ConnectX-7

~# mlxconfig -d 0d:00.0 q

Device #1:

Device type: ConnectX7
Name: MCX75310AAS-NEA_Ax
Description: NVIDIA ConnectX-7 HHHL Adapter card; 400GbE / NDR IB (default mode); Single-port OSFP; PCIe 5.0 x16; Crypto Disabled; Secure Boot Enabled; Device: 0d:00.0


## Actual behavior:

```bash
~# LD_PRELOAD=libxlio-debug.so sockperf sr -i 172.17.13.50
 XLIO INFO   : ---------------------------------------------------------------------------
 XLIO INFO   : XLIO_VERSION: 3.40.2-1 Release built on Oct  7 2024 17:14:06 -*- DEBUG -*-
 XLIO INFO   : Cmd Line: sockperf sr -i 172.17.13.50
 XLIO INFO   : OFED Version: MLNX_OFED_LINUX-24.10-0.7.0.0:
 XLIO INFO   : ---------------------------------------------------------------------------
 XLIO INFO   : Log Level                      INFO                       [XLIO_TRACELEVEL]
 XLIO INFO   : ---------------------------------------------------------------------------
 XLIO WARNING: ************************************************************************
 XLIO WARNING: Your current max locked memory is: 16878186496. Please change it to unlimited.
 XLIO WARNING: Set this user's default to `ulimit -l unlimited`.
 XLIO WARNING: Read more about this topic in the XLIO's User Manual.
 XLIO WARNING: ************************************************************************
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/cali37b21ebe0db/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev94/address (errno 2 No such file or directory)
 XLIO ERROR  : utils:552:priv_read_file() Couldn't open file /sys/class/net/dev99/address (errno 2 No such file or directory)

It looks like it's looping forever and my sockperf tests are not succeeding.

~# xlio_stats 
xliostat: stats for application: sockperf with pid: 1717047
 fd      ------------ total offloaded -------------   --------- total os ----------
                pkt   Kbyte   eagain   error  poll%     pkt   Kbyte  eagain   error

Expected behavior:

The sockperf test can see the acceleration effect.

Steps to reproduce:

server:

LD_PRELOAD=libxlio.so sockperf sr -i 172.17.13.50

client:

LD_PRELOAD=libxlio.so sockperf pp -i 172.17.13.50

ty-dc commented 13 hours ago

@AlexanderGrissik @pasis @alexbriskin @tomerdbz Sorry to bother you, I have a problem with the xlio program, can you give me some advice?