search

jwbensley / Etherate

Linux CLI Ethernet and MPLS Testing Tool
MIT License
175 stars 19 forks source link

X710 10G interface produces "No buffer space available" error. #40

Open jwbensley opened 6 years ago

jwbensley commented 6 years ago

ISSUE TYPE

VERSION DETAILS

$ uname -a
Linux ucpe_002 4.13.12-1.el7.elrepo.x86_64 #1 SMP Wed Nov 8 13:01:08 EST 2017 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/redhat-release || lsb_release -a || cat /etc/issue
CentOS Linux release 7.4.1708 (Core)

$ lspci -nn | grep -i eth
02:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
02:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
06:00.0 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
06:00.1 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
06:00.2 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
06:00.3 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
09:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
09:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens2f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq portid 1402ec6d9dd0 state UP mode DEFAULT qlen 1000
    link/ether 14:02:ec:6d:9d:d0 brd ff:ff:ff:ff:ff:ff
3: ens3f0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:f8 brd ff:ff:ff:ff:ff:ff
4: ens3f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:f9 brd ff:ff:ff:ff:ff:ff
5: ens2f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq portid 1402ec6d9dd8 state UP mode DEFAULT qlen 10000
    link/ether 14:02:ec:6d:9d:d8 brd ff:ff:ff:ff:ff:ff
6: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 94:18:82:ab:ae:f3 brd ff:ff:ff:ff:ff:ff
7: ens3f2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fa brd ff:ff:ff:ff:ff:ff
8: ens3f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
9: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
    link/ether 94:18:82:ab:ae:f4 brd ff:ff:ff:ff:ff:ff
10: ens3f3.501@ens3f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br501 state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
11: br501: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
12: ens3f3.502@ens3f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br502 state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
13: br502: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000
    link/ether 3c:a8:2a:e7:05:fb brd ff:ff:ff:ff:ff:ff
14: br0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
15: br1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
16: br2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1000
    link/ether 02:84:2b:68:80:af brd ff:ff:ff:ff:ff:ff
17: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:41:c6:5e brd ff:ff:ff:ff:ff:ff
18: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:41:c6:5e brd ff:ff:ff:ff:ff:ff

$ ./etherate -V
Etherate version 1.15 2017-10

$ sudo ./etherate -l
Device lo with address 00:00:00:00:00:00, has interface index 1
Device ens2f0 with address 14:02:ec:6d:9d:d0, has interface index 2
Device ens3f0 with address 3c:a8:2a:e7:05:f8, has interface index 3
Device ens3f1 with address 3c:a8:2a:e7:05:f9, has interface index 4
Device ens2f1 with address 14:02:ec:6d:9d:d8, has interface index 5
Device eno1 with address 94:18:82:ab:ae:f3, has interface index 6
Device ens3f2 with address 3c:a8:2a:e7:05:fa, has interface index 7
Device ens3f3 with address 3c:a8:2a:e7:05:fb, has interface index 8
Device eno2 with address 94:18:82:ab:ae:f4, has interface index 9
Device ens3f3.501 with address 3c:a8:2a:e7:05:fb, has interface index 10
Device br501 with address 3c:a8:2a:e7:05:fb, has interface index 11
Device ens3f3.502 with address 3c:a8:2a:e7:05:fb, has interface index 12
Device br502 with address 3c:a8:2a:e7:05:fb, has interface index 13
Device br0 with address 00:00:00:00:00:00, has interface index 14
Device br1 with address 00:00:00:00:00:00, has interface index 15
Device br2 with address 02:84:2b:68:80:af, has interface index 16
Device virbr0 with address 52:54:00:41:c6:5e, has interface index 17
Device virbr0-nic with address 52:54:00:41:c6:5e, has interface index 18

SUMMARY Only on the two 10G interfaces is this issue experianced, not on the 1G interfaces. Does increasing the socket buffer help?

STEPS TO REPRODUCE This is happeneing at all packet sizes, 256 is not specific to this issue:

$ sudo ./etherate -i ens2f1 -g -G -f 256
Running in TX mode
Using device ens2f1 with address 14:02:ec:6d:9d:d8, interface index 5
Entering promiscuous mode
Source MAC 00:00:5e:00:00:01
Destination MAC 00:00:5e:00:00:02
Sending gratuitous broadcasts...
Done.
Frame size is 270 bytes
Starting test on Mon Nov 13 17:03:13 2017

Seconds         Mbps Tx         MBs Tx          FrmTx/s         Frames Tx
1               914.41          109             423336          423336
2               1174.33         248             543672          967008
3               1174.70         389             543841          1510849
4               1174.62         529             543805          2054654
Speed test Tx error : No buffer space available
Ending test on Mon Nov 13 17:03:17 2017

Leaving promiscuous mode

This issue doesn't occur using the 1G interface, which was tested using various packet sizes. However it does seem to be happening on the 10G interfaces at any packet size I try.

jwbensley commented 6 years ago

Socket is hitting ENOBUFS, need to ignore this errno rather than dying. stall and carry on.