search

emmericp / MoonGen

MoonGen is a fully scriptable high-speed packet generator built on DPDK and LuaJIT. It can saturate a 10 Gbit/s connection with 64 byte packets on a single CPU core while executing user-provided Lua scripts for each packet. Multi-core support allows for even higher rates. It also features precise and accurate timestamping and rate control.
MIT License
1.05k stars 235 forks source link

Chelsio ports are not found #295

Open davidgengenbach opened 3 years ago

davidgengenbach commented 3 years ago

I have problems getting Chelsio T62100-SO-CR to work with DPDK/MoonGen. Could you please provide some hints for troubleshooting? Thanks in advance!

I found these pages but had no luck testing the approaches:

I know the distribution is a little old - unfortunately, I can not update that easily...

$ cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.6 LTS"
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
$ sudo testpmd
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 2 on socket 0
EAL: Detected lcore 2 as core 4 on socket 0
EAL: Detected lcore 3 as core 9 on socket 2
EAL: Detected lcore 4 as core 11 on socket 2
EAL: Detected lcore 5 as core 0 on socket 2
EAL: Detected lcore 6 as core 2 on socket 2
EAL: Detected lcore 7 as core 4 on socket 2
EAL: Detected lcore 8 as core 9 on socket 0
EAL: Detected lcore 9 as core 11 on socket 0
EAL: Detected lcore 10 as core 1 on socket 0
EAL: Detected lcore 11 as core 3 on socket 0
EAL: Detected lcore 12 as core 8 on socket 2
EAL: Detected lcore 13 as core 10 on socket 2
EAL: Detected lcore 14 as core 12 on socket 2
EAL: Detected lcore 15 as core 1 on socket 2
EAL: Detected lcore 16 as core 3 on socket 2
EAL: Detected lcore 17 as core 8 on socket 0
EAL: Detected lcore 18 as core 10 on socket 0
EAL: Detected lcore 19 as core 12 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 20 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Setting up physically contiguous memory...
EAL: Ask a virtual area of 0x7c00000 bytes
EAL: Virtual area found at 0x7fe8bac00000 (size = 0x7c00000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe8ba800000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe8ba400000 (size = 0x200000)
EAL: Ask a virtual area of 0x2e800000 bytes
EAL: Virtual area found at 0x7fe88ba00000 (size = 0x2e800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe88b600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe88b200000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fe88ac00000 (size = 0x400000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fe88a600000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe88a200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe889e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x600000 bytes
EAL: Virtual area found at 0x7fe889600000 (size = 0x600000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe889200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe888e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe888a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe888600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe888200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe887e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe887a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe887600000 (size = 0x200000)
EAL: Ask a virtual area of 0x1400000 bytes
EAL: Virtual area found at 0x7fe886000000 (size = 0x1400000)
EAL: Ask a virtual area of 0x600000 bytes
EAL: Virtual area found at 0x7fe885800000 (size = 0x600000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe885400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe885000000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe884c00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe884800000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe884400000 (size = 0x200000)
EAL: Ask a virtual area of 0x1200000 bytes
EAL: Virtual area found at 0x7fe883000000 (size = 0x1200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe882c00000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fe882600000 (size = 0x400000)
EAL: Ask a virtual area of 0x1800000 bytes
EAL: Virtual area found at 0x7fe880c00000 (size = 0x1800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe880800000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe880400000 (size = 0x200000)
EAL: Ask a virtual area of 0xa00000 bytes
EAL: Virtual area found at 0x7fe87f800000 (size = 0xa00000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7fe87ee00000 (size = 0x800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe87ea00000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fe87e400000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe87e000000 (size = 0x200000)
EAL: Ask a virtual area of 0x37800000 bytes
EAL: Virtual area found at 0x7fe846600000 (size = 0x37800000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7fe845c00000 (size = 0x800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe845800000 (size = 0x200000)
EAL: Ask a virtual area of 0x1000000 bytes
EAL: Virtual area found at 0x7fe844600000 (size = 0x1000000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fe844000000 (size = 0x400000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fe843a00000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe843600000 (size = 0x200000)
EAL: Ask a virtual area of 0x600000 bytes
EAL: Virtual area found at 0x7fe842e00000 (size = 0x600000)
EAL: Ask a virtual area of 0xa00000 bytes
EAL: Virtual area found at 0x7fe842200000 (size = 0xa00000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7fe841800000 (size = 0x800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe841400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe841000000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fe840a00000 (size = 0x400000)
EAL: Ask a virtual area of 0xc00000 bytes
EAL: Virtual area found at 0x7fe83fc00000 (size = 0xc00000)
EAL: Ask a virtual area of 0xa00000 bytes
EAL: Virtual area found at 0x7fe83f000000 (size = 0xa00000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7fe83ea00000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe83e600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fe83e200000 (size = 0x200000)
EAL: Ask a virtual area of 0x1000000 bytes
EAL: Virtual area found at 0x7fe83d000000 (size = 0x1000000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7fe83c600000 (size = 0x800000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7fe83bc00000 (size = 0x800000)
EAL: Ask a virtual area of 0x600000 bytes
EAL: Virtual area found at 0x7fe83b400000 (size = 0x600000)
EAL: Requesting 512 pages of size 2MB from socket 0
EAL: Requesting 512 pages of size 2MB from socket 2
EAL: TSC frequency is ~2297308 KHz
EAL: Master lcore 0 is ready (tid=c5cac8c0;cpuset=[0])
EAL: lcore 10 is ready (tid=36bf6700;cpuset=[10])
EAL: lcore 11 is ready (tid=363f5700;cpuset=[11])
EAL: lcore 1 is ready (tid=3b3ff700;cpuset=[1])
EAL: lcore 8 is ready (tid=37bf8700;cpuset=[8])
EAL: lcore 9 is ready (tid=373f7700;cpuset=[9])
EAL: lcore 6 is ready (tid=38bfa700;cpuset=[6])
EAL: lcore 4 is ready (tid=39bfc700;cpuset=[4])
EAL: lcore 13 is ready (tid=353f3700;cpuset=[13])
EAL: lcore 3 is ready (tid=3a3fd700;cpuset=[3])
EAL: lcore 7 is ready (tid=383f9700;cpuset=[7])
EAL: lcore 12 is ready (tid=35bf4700;cpuset=[12])
EAL: lcore 17 is ready (tid=effd700;cpuset=[17])
EAL: lcore 5 is ready (tid=393fb700;cpuset=[5])
EAL: lcore 19 is ready (tid=dffb700;cpuset=[19])
EAL: lcore 15 is ready (tid=ffff700;cpuset=[15])
EAL: lcore 16 is ready (tid=f7fe700;cpuset=[16])
EAL: lcore 18 is ready (tid=e7fc700;cpuset=[18])
EAL: lcore 2 is ready (tid=3abfe700;cpuset=[2])
EAL: lcore 14 is ready (tid=34bf2700;cpuset=[14])
EAL: No probed ethernet devices
PMD: gntalloc: ioctl error
Done
No commandline core given, start packet forwarding
  io packet forwarding - CRC stripping disabled - packets/burst=32
  nb forwarding cores=1 - nb forwarding ports=0
  RX queues=1 - RX desc=128 - RX free threshold=0
  RX threshold registers: pthresh=0 hthresh=0 wthresh=0
  TX queues=1 - TX desc=512 - TX free threshold=0
  TX threshold registers: pthresh=0 hthresh=0 wthresh=0
  TX RS bit threshold=0 - TXQ flags=0x0
Press enter to exit

Telling cores to stop...
Waiting for lcores to finish...

  +++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
  RX-packets: 0              RX-dropped: 0             RX-total: 0
  TX-packets: 0              TX-dropped: 0             TX-total: 0
  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done.

Bye...
$ $ sudo MoonGen/libmoon/deps/dpdk/usertools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver
============================================
0000:05:00.0 'T62100-SO-CR Unified Wire Ethernet Controller 6008' drv=igb_uio unused=vfio-pci,uio_pci_generic

Network devices using kernel driver
===================================
0000:02:00.0 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno1 drv=tg3 unused=igb_uio,vfio-pci,uio_pci_generic *Active*
0000:02:00.1 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno2 drv=tg3 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:02:00.2 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno3 drv=tg3 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:02:00.3 'NetXtreme BCM5719 Gigabit Ethernet PCIe 1657' if=eno4 drv=tg3 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:04:00.0 'NetXtreme II BCM57810 10 Gigabit Ethernet 168e' if=eno49 drv=bnx2x unused=igb_uio,vfio-pci,uio_pci_generic 
0000:04:00.1 'NetXtreme II BCM57810 10 Gigabit Ethernet 168e' if=eno50 drv=bnx2x unused=igb_uio,vfio-pci,uio_pci_generic 
0000:05:00.1 'T62100-SO-CR Unified Wire Ethernet Controller 6008' if= drv=cxgb4 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:05:00.2 'T62100-SO-CR Unified Wire Ethernet Controller 6008' if= drv=cxgb4 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:05:00.3 'T62100-SO-CR Unified Wire Ethernet Controller 6008' if= drv=cxgb4 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:0b:00.0 'T62100-SO-CR Unified Wire Ethernet Controller 6008' if= drv=cxgb4 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:0b:00.1 'T62100-SO-CR Unified Wire Ethernet Controller 6008' if= drv=cxgb4 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:0b:00.2 'T62100-SO-CR Unified Wire Ethernet Controller 6008' if= drv=cxgb4 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:0b:00.3 'T62100-SO-CR Unified Wire Ethernet Controller 6008' if= drv=cxgb4 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:0b:00.4 'T62100-SO-CR Unified Wire Ethernet Controller 6408' if=ens2f4,ens2f4d1 drv=cxgb4 unused=igb_uio,vfio-pci,uio_pci_generic 
0000:81:00.0 'MT27520 Family [ConnectX-3 Pro] 1007' if=ens6,ens6d1 drv=mlx4_core unused=igb_uio,vfio-pci,uio_pci_generic *Active*

Other Network devices
=====================
0000:05:00.4 'T62100-SO-CR Unified Wire Ethernet Controller 6408' unused=igb_uio,vfio-pci,uio_pci_generic

Crypto devices using DPDK-compatible driver
===========================================
<none>

Crypto devices using kernel driver
==================================
<none>

Other Crypto devices
====================
<none>

Eventdev devices using DPDK-compatible driver
=============================================
<none>

Eventdev devices using kernel driver
====================================
<none>

Other Eventdev devices
======================
<none>

Mempool devices using DPDK-compatible driver
============================================
<none>

Mempool devices using kernel driver
===================================
<none>

Other Mempool devices
=====================
<none>
$ lsmod
Module                  Size  Used by
cxgb4                 671744  0
geneve                 28672  1 cxgb4
uio_pci_generic        16384  0
nfnetlink_queue        20480  0
nfnetlink_log          20480  0
bluetooth             520192  0
igb_uio                16384  0
uio                    20480  2 uio_pci_generic,igb_uio
vfio_pci               40960  0
vfio_iommu_type1       20480  0
vfio_virqfd            16384  1 vfio_pci
vfio                   28672  2 vfio_iommu_type1,vfio_pci
xt_CHECKSUM            16384  1
iptable_mangle         16384  1
ipt_REJECT             16384  2
nf_reject_ipv4         16384  1 ipt_REJECT
xt_tcpudp              16384  6
ebtable_filter         16384  0
ebtables               32768  1 ebtable_filter
ipt_MASQUERADE         16384  4
nf_nat_masquerade_ipv4    16384  1 ipt_MASQUERADE
nf_conntrack_netlink    40960  0
nfnetlink              16384  4 nfnetlink_log,nf_conntrack_netlink,nfnetlink_queue
xfrm_user              32768  1
xfrm_algo              16384  1 xfrm_user
iptable_nat            16384  1
nf_conntrack_ipv4      20480  3
nf_defrag_ipv4         16384  1 nf_conntrack_ipv4
nf_nat_ipv4            16384  1 iptable_nat
xt_addrtype            16384  2
xt_conntrack           16384  2
nf_nat                 28672  2 nf_nat_ipv4,nf_nat_masquerade_ipv4
nf_conntrack          106496  6 nf_nat,nf_nat_ipv4,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_netlink,nf_conntrack_ipv4
br_netfilter           24576  0
bridge                126976  1 br_netfilter
stp                    16384  1 bridge
llc                    16384  2 stp,bridge
pci_stub               16384  1
vboxpci                24576  0
vboxnetadp             28672  0
vboxnetflt             28672  0
vboxdrv               458752  3 vboxnetadp,vboxnetflt,vboxpci
xt_multiport           16384  4
iptable_filter         16384  1
ip_tables              24576  3 iptable_filter,iptable_mangle,iptable_nat
ip6table_filter        16384  1
ip6_tables             28672  1 ip6table_filter
x_tables               36864  13 ip6table_filter,xt_CHECKSUM,ip_tables,xt_tcpudp,ipt_MASQUERADE,xt_conntrack,xt_multiport,iptable_filter,ebtables,ipt_REJECT,iptable_mangle,ip6_tables,xt_addrtype
aufs                  217088  0
overlay                49152  0
mos7840                40960  0
cp210x                 24576  0
usbserial              53248  2 cp210x,mos7840
binfmt_misc            20480  1
ipmi_ssif              24576  0
intel_rapl             20480  0
x86_pkg_temp_thermal    16384  0
intel_powerclamp       16384  0
coretemp               16384  0
joydev                 20480  0
input_leds             16384  0
sb_edac                32768  0
hpilo                  20480  0
edac_core              53248  1 sb_edac
lpc_ich                24576  0
ioatdma                53248  0
shpchp                 36864  0
dca                    16384  1 ioatdma
ipmi_si                57344  0
ipmi_msghandler        49152  2 ipmi_ssif,ipmi_si
8250_fintek            16384  0
kvm                   548864  0
acpi_power_meter       20480  0
mac_hid                16384  0
irqbypass              16384  2 kvm,vfio_pci
ib_iser                49152  0
rdma_cm                49152  1 ib_iser
iw_cm                  45056  1 rdma_cm
ib_cm                  45056  1 rdma_cm
ib_sa                  36864  2 rdma_cm,ib_cm
ib_mad                 49152  2 ib_cm,ib_sa
ib_core               106496  6 rdma_cm,ib_cm,ib_sa,iw_cm,ib_mad,ib_iser
ib_addr                20480  2 rdma_cm,ib_core
iscsi_tcp              20480  0
libiscsi_tcp           24576  1 iscsi_tcp
libiscsi               53248  3 libiscsi_tcp,iscsi_tcp,ib_iser
scsi_transport_iscsi   102400  4 iscsi_tcp,ib_iser,libiscsi
parport_pc             32768  0
ppdev                  20480  0
lp                     20480  0
parport                49152  3 lp,ppdev,parport_pc
autofs4                40960  2
btrfs                 991232  0
raid10                 49152  0
raid456               110592  0
async_raid6_recov      20480  1 raid456
async_memcpy           16384  2 raid456,async_raid6_recov
async_pq               16384  2 raid456,async_raid6_recov
async_xor              16384  3 async_pq,raid456,async_raid6_recov
async_tx               16384  5 async_pq,raid456,async_xor,async_memcpy,async_raid6_recov
xor                    24576  2 btrfs,async_xor
raid6_pq              102400  4 async_pq,raid456,btrfs,async_raid6_recov
raid1                  36864  0
raid0                  20480  0
multipath              16384  0
linear                 16384  0
mlx4_en               110592  0
crct10dif_pclmul       16384  0
crc32_pclmul           16384  0
ghash_clmulni_intel    16384  0
aesni_intel           167936  0
bnx2x                 724992  0
aes_x86_64             20480  1 aesni_intel
hid_generic            16384  0
lrw                    16384  1 aesni_intel
vxlan                  49152  3 bnx2x,cxgb4,mlx4_en
gf128mul               16384  1 lrw
usbhid                 49152  0
ip6_udp_tunnel         16384  2 vxlan,geneve
tg3                   167936  0
udp_tunnel             16384  2 vxlan,geneve
glue_helper            16384  1 aesni_intel
ablk_helper            16384  1 aesni_intel
ahci                   36864  0
ptp                    20480  4 tg3,bnx2x,cxgb4,mlx4_en
mdio                   16384  1 bnx2x
cryptd                 20480  3 ghash_clmulni_intel,aesni_intel,ablk_helper
mlx4_core             286720  1 mlx4_en
hpsa                   98304  3
libahci                32768  1 ahci
hid                   122880  2 hid_generic,usbhid
pps_core               20480  1 ptp
libcrc32c              16384  2 raid456,bnx2x
scsi_transport_sas     45056  1 hpsa
wmi                    20480  0
fjes                   28672  0
$ sudo lspci -vvv -s 0000:05:00.0
05:00.0 Ethernet controller: Chelsio Communications Inc T62100-SO-CR Unified Wire Ethernet Controller
    Subsystem: Chelsio Communications Inc Device 0000
    Physical Slot: 1
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 16
    Region 0: Memory at 96580000 (64-bit, non-prefetchable) [size=512K]
    Region 2: Memory at 96600000 (64-bit, non-prefetchable) [size=512K]
    Region 4: Memory at 9690c000 (64-bit, non-prefetchable) [size=8K]
    [virtual] Expansion ROM at 96980000 [disabled] [size=512K]
    Capabilities: [40] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [50] MSI: Enable- Count=1/8 Maskable+ 64bit+
        Address: 0000000000000000  Data: 0000
        Masking: 00000000  Pending: 00000000
    Capabilities: [70] Express (v2) Endpoint, MSI 00
        DevCap: MaxPayload 2048 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
        DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset-
            MaxPayload 256 bytes, MaxReadReq 4096 bytes
        DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
        LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM not supported, Exit Latency L0s <256ns, L1 <64us
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed 8GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, OBFF Not Supported
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
        LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+
             EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-
    Capabilities: [b0] MSI-X: Enable+ Count=34 Masked-
        Vector table: BAR=4 offset=00000000
        PBA: BAR=4 offset=00001000
    Capabilities: [d0] Vital Product Data
        Product Name: T62100-SO-CR    
        Read-only fields:
            [PN] Part number: 110122660B0     
            [EC] Engineering changes: 0000000000000000
            [SN] Serial number: PT24200017              
            [RV] Reserved: checksum good, 165 byte(s) reserved
        Read/write fields:
            [RW] Read-write area: 249 byte(s) free
        End
    Capabilities: [100 v2] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
        AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
    Capabilities: [148 v1] Virtual Channel
        Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
        Arb:    Fixed- WRR32- WRR64- WRR128-
        Ctrl:   ArbSelect=Fixed
        Status: InProgress-
        VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
            Status: NegoPending- InProgress-
    Capabilities: [168 v1] Device Serial Number 00-00-00-00-00-00-00-00
    Capabilities: [188 v1] Alternative Routing-ID Interpretation (ARI)
        ARICap: MFVC- ACS-, Next Function: 1
        ARICtl: MFVC- ACS-, Function Group: 0
    Capabilities: [198 v1] #19
    Capabilities: [1c8 v1] Single Root I/O Virtualization (SR-IOV)
        IOVCap: Migration-, Interrupt Message Number: 000
        IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+
        IOVSta: Migration-
        Initial VFs: 16, Total VFs: 16, Number of VFs: 0, Function Dependency Link: 00
        VF offset: 8, stride: 4, Device ID: 6808
        Supported Page Size: 00000553, System Page Size: 00000001
        Region 0: Memory at 000000009690e000 (64-bit, non-prefetchable)
        Region 2: Memory at 0000000096680000 (64-bit, non-prefetchable)
        Region 4: Memory at 0000000096880000 (64-bit, non-prefetchable)
        VF Migration: offset: 00000000, BIR: 0
    Capabilities: [208 v1] Transaction Processing Hints
        Interrupt vector mode supported
        Steering table in MSI-X table
    Kernel driver in use: igb_uio
    Kernel modules: cxgb4
$ dmesg | grep cxgb
[ 3086.159872] cxgb4 0000:0b:00.4: Coming up as MASTER: Initializing adapter
[ 3087.985611] cxgb4 0000:0b:00.4: Reserved 92MB host memory for HMA
[ 3087.988742] cxgb4 0000:0b:00.4: Successfully enabled ppod edram feature
[ 3088.749994] cxgb4 0000:0b:00.4: Successfully configured using Firmware Configuration File "/lib/firmware/cxgb4/t6-config.txt", version 0x1000033, computed checksum 0xbcf115d
[ 3088.779423] cxgb4 0000:0b:00.4: max_ordird_qp 21 max_ird_adapter 5376
[ 3088.783637] cxgb4 0000:0b:00.4: ppod edram start 0x0 end 0x0 size 0x1
[ 3088.786874] cxgb4 0000:0b:00.4: Current filter mode/mask 0x632b:0x21
[ 3088.800964] cxgb4 0000:0b:00.4: Registering cxgb4 panic handler.., Buffer start address = ffffc9000ac81000, size: 33554432
[ 3088.809368] cxgb4 0000:0b:00.4: adap_smt_index: check the config file for nsmt (0)
[ 3088.813027] cxgb4 0000:0b:00.4: 145 MSI-X vectors allocated, nic 32 ofld 32 rdma cpl 2 rdma ciq 28 iscsi 12 iscsit 20
[ 3088.813031] cxgb4 0000:0b:00.4:  crypto rx 16
[ 3088.827646] cxgb4 0000:0b:00.4: PCIe link speed is 8.0GT/s, device supports 8.0GT/s
[ 3088.827653] cxgb4 0000:0b:00.4: PCIe link width is x8, device supports x16
[ 3088.827657] cxgb4 0000:0b:00.4: A slot with more lanes and/or higher speed is suggested for optimal performance.
[ 3088.829871] cxgb4 0000:0b:00.4: Chelsio T62100-SO-CR rev 0
[ 3088.829873] cxgb4 0000:0b:00.4: S/N: PT24200035, P/N: 110122660B0
[ 3088.829876] cxgb4 0000:0b:00.4: Firmware version: 1.24.17.0
[ 3088.829878] cxgb4 0000:0b:00.4: Bootstrap version: 255.255.255.255
[ 3088.829880] cxgb4 0000:0b:00.4: TP Microcode version: 0.1.23.2
[ 3088.829882] cxgb4 0000:0b:00.4: Expansion ROM version: 2.0.0.8
[ 3088.829884] cxgb4 0000:0b:00.4: Serial Configuration version: 0x8002000
[ 3088.829886] cxgb4 0000:0b:00.4: VPD version: 0x83
[ 3088.829888] cxgb4 0000:0b:00.4: Configuration: RNIC MSI-X, Offload capable
[ 3088.829951] cxgb4 0000:0b:00.4 ens2f4: renamed from eth0
[ 3088.851913] cxgb4 0000:0b:00.4 ens2f4d1: renamed from eth1
davidgengenbach commented 3 years ago

I think I found the problem (mentioned in the issue): Chelsio does not support DPDK for this specific Ubuntu version (16.04 LTS)...

davidgengenbach commented 3 years ago

No, was not the cause :( Still getting the same errors with Ubuntu 20.04 LTS