search

F-Stack / f-stack

F-Stack is an user space network development kit with high performance based on DPDK, FreeBSD TCP/IP stack and coroutine API.
http://www.f-stack.org
Other
3.86k stars 897 forks source link

No probed ethernet devices #531

Open Sdarot opened 4 years ago

Sdarot commented 4 years ago

When I try to run ff_start, to test the example "hello world" app, I'm facing an error "No probed ethernet devices"

running CentOS Linux release 7.8.2003 (Core) f-stack (dpdk) compiled with CONFIG_RTE_LIBRTE_MLX5_PMD=y

ff_start -c /etc/f-stack.conf

./example/helloworld --conf /etc/f-stack.conf --proc-type=primary --proc-id=0
net_mlx5: cannot load glue library: librte_pmd_mlx5_glue.so.19.08.0: cannot open shared object file: No such file or directory
net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5)
[dpdk]: lcore_mask=1
[dpdk]: channel=4
[dpdk]: promiscuous=1
[dpdk]: numa_on=0
[dpdk]: tx_csum_offoad_skip=0
[dpdk]: tso=0
[dpdk]: vlan_strip=1
[dpdk]: idle_sleep=0
[dpdk]: pkt_tx_delay=100
[dpdk]: port_list=0
[dpdk]: nb_vdev=0
[dpdk]: nb_bond=0
[pcap]: enable=0
[pcap]: snaplen=96
[pcap]: savelen=16777216
[port0]: addr=192.168.0.11
[port0]: netmask=255.255.225.0
[port0]: broadcast=192.168.0.255
[port0]: gateway=192.168.0.1
[kni]: enable=0
[kni]: method=reject
[kni]: tcp_port=80,443
[kni]: udp_port=53
[freebsd.boot]: hz=100
[freebsd.boot]: fd_reserve=1024
[freebsd.boot]: kern.ipc.maxsockets=262144
[freebsd.boot]: net.inet.tcp.syncache.hashsize=4096
[freebsd.boot]: net.inet.tcp.syncache.bucketlimit=100
[freebsd.boot]: net.inet.tcp.tcbhashsize=65536
[freebsd.boot]: kern.ncallout=262144
[freebsd.boot]: kern.features.inet6=1
[freebsd.boot]: net.inet6.ip6.auto_linklocal=1
[freebsd.boot]: net.inet6.ip6.accept_rtadv=2
[freebsd.boot]: net.inet6.icmp6.rediraccept=1
[freebsd.boot]: net.inet6.ip6.forwarding=0
[freebsd.sysctl]: kern.ipc.somaxconn=32768
[freebsd.sysctl]: kern.ipc.maxsockbuf=16777216
[freebsd.sysctl]: net.link.ether.inet.maxhold=5
[freebsd.sysctl]: net.inet.tcp.fast_finwait2_recycle=1
[freebsd.sysctl]: net.inet.tcp.sendspace=16384
[freebsd.sysctl]: net.inet.tcp.recvspace=8192
[freebsd.sysctl]: net.inet.tcp.cc.algorithm=cubic
[freebsd.sysctl]: net.inet.tcp.sendbuf_max=16777216
[freebsd.sysctl]: net.inet.tcp.recvbuf_max=16777216
[freebsd.sysctl]: net.inet.tcp.sendbuf_auto=1
[freebsd.sysctl]: net.inet.tcp.recvbuf_auto=1
[freebsd.sysctl]: net.inet.tcp.sendbuf_inc=16384
[freebsd.sysctl]: net.inet.tcp.recvbuf_inc=524288
[freebsd.sysctl]: net.inet.tcp.sack.enable=1
[freebsd.sysctl]: net.inet.tcp.blackhole=1
[freebsd.sysctl]: net.inet.tcp.msl=2000
[freebsd.sysctl]: net.inet.tcp.delayed_ack=0
[freebsd.sysctl]: net.inet.udp.blackhole=1
[freebsd.sysctl]: net.inet.ip.redirect=0
[freebsd.sysctl]: net.inet.ip.forwarding=0
EAL: Detected 24 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Forcing IOVA as 'PA' because KNI module is loaded
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
f-stack -c1 -n4 --proc-type=primary EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:00:04.0 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.1 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.2 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.3 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.4 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.5 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.6 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.7 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:1b:00.0 on NUMA socket 0
EAL:   probe driver: 8086:37d2 net_i40e
EAL: PCI device 0000:1b:00.1 on NUMA socket 0
EAL:   probe driver: 8086:37d2 net_i40e
EAL: Error - exiting with code: 1
  Cause: No probed ethernet devices

uname -a Linux dev 4.14.35-1902.304.6.el7uek.x86_64 #2 SMP Thu Jul 2 18:16:54 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux

dpdk-devbind -s

Network devices using DPDK-compatible driver
============================================
0000:b3:00.0 'MT27800 Family [ConnectX-5] 1017' drv=igb_uio unused=mlx5_core

Network devices using kernel driver
===================================
0000:1b:00.0 'Ethernet Connection X722 for 10GBASE-T 37d2' if=eno1 drv=i40e unused=igb_uio *Active*
0000:1b:00.1 'Ethernet Connection X722 for 10GBASE-T 37d2' if=eno2 drv=i40e unused=igb_uio
0000:b3:00.1 'MT27800 Family [ConnectX-5] 1017' if=enp179s0f1 drv=mlx5_core unused=igb_uio

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

Misc (rawdev) devices using kernel driver
=========================================
0000:00:04.0 'Sky Lake-E CBDMA Registers 2021' if= drv=ioatdma unused=igb_uio
0000:00:04.1 'Sky Lake-E CBDMA Registers 2021' if= drv=ioatdma unused=igb_uio
0000:00:04.2 'Sky Lake-E CBDMA Registers 2021' if= drv=ioatdma unused=igb_uio
0000:00:04.3 'Sky Lake-E CBDMA Registers 2021' if= drv=ioatdma unused=igb_uio
0000:00:04.4 'Sky Lake-E CBDMA Registers 2021' if= drv=ioatdma unused=igb_uio
0000:00:04.5 'Sky Lake-E CBDMA Registers 2021' if= drv=ioatdma unused=igb_uio
0000:00:04.6 'Sky Lake-E CBDMA Registers 2021' if= drv=ioatdma unused=igb_uio
0000:00:04.7 'Sky Lake-E CBDMA Registers 2021' if= drv=ioatdma unused=igb_uio

libraries: libibverbs-22.4-2.el7_8.x86_64 rdma-core-devel-22.4-2.el7_8.x86_64

jfb8856606 commented 4 years ago

F-Stack‘s guide not applicable to Mellanox NIC, you should flow dpdk's guide or Mellanox's guide.

Sdarot commented 4 years ago

I finally solved this problem. You should copy librte_pmd_mlx5_glue.so.* to /lib64 directory

ThreeWater1999 commented 2 years ago

I finally solved this problem. You should copy librte_pmd_mlx5_glue.so.* to /lib64 directory

Can you explain more concretely? Thanks a lot. My problem is that DPDK can successfully use Mellanox NIC，but F-Stack can't. The example is both helloworld.

F-Stack is as follow

proc_lcore[0] = 0

port id = 0; nb_lcores= 1

EAL: Detected 8 lcore(s) EAL: Detected 1 NUMA nodes EAL: Auto-detected process type: PRIMARY EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' EAL: No available hugepages reported in hugepages-1048576kB f-stack -c1 -n4 --proc-type=auto EAL: Probing VFIO support... EAL: PCI device 0000:00:1f.6 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:15bc net_e1000_em EAL: PCI device 0000:03:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:1583 net_i40e EAL: PCI device 0000:03:00.1 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:1583 net_i40e EAL: Error - exiting with code: 1 Cause: No probed ethernet devices

DPDK is as follow EAL: PCI device 0000:00:1f.6 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:15bc net_e1000_em EAL: PCI device 0000:01:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1017 net_mlx5 EAL: PCI device 0000:01:00.1 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1017 net_mlx5 EAL: PCI device 0000:03:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:1583 net_i40e EAL: PCI device 0000:03:00.1 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 8086:1583 net_i40e

Mellanox NIC‘s PCI is 0000:01:00.0 and 0000:01:00.1, DPDK can probe it, but F-Stack can't. Can you give me some advice? Thanks a lot!