Unable to probe PCI devices within rte_eal_init() on onvm_mgr start

[rudh24]

Bug Report

Current Behavior The onvm_mgr does not initiate the PCI probe within the rte_eal_init(). This ends up reporting 0 eth devices, and subsequently no ports to work with. I've followed the installation steps listed. The only difference is that I work with Mellanox NICs.

The DPDK helloworld example appears to work. Please see "Additional Context" for the console output.

Expected behavior/code onvm_mgr startup should find the available devices.

Steps to reproduce Follow the steps in https://github.com/sdnfv/openNetVM/blob/master/docs/Install.md

Environment

• OS: ubuntu 16.04
• onvm version: 20.10
• dpdk version: 20.05
• other info:

Possible Solution

I found a DPDK mailing thread where people faced a similar issue: https://mails.dpdk.org/archives/users/2016-March/000375.html But the fix mentioned in this hasn't worked for me so far.

Additional context/Screenshots Hello world output.

$ sudo ./build/helloworld -l 0,1,2 -n 4 --proc-type=primary
EAL: Detected 24 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: PCI device 0000:00:04.0 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.1 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.2 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.3 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.4 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.5 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.6 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:00:04.7 on NUMA socket 0
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:18:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1563 net_ixgbe
EAL: PCI device 0000:18:00.1 on NUMA socket 0
EAL:   probe driver: 8086:1563 net_ixgbe
EAL: PCI device 0000:3b:00.0 on NUMA socket 0
EAL:   probe driver: 15b3:101d net_mlx5
EAL: PCI device 0000:3b:00.1 on NUMA socket 0
EAL:   probe driver: 15b3:101d net_mlx5
EAL: PCI device 0000:80:04.0 on NUMA socket 1
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:80:04.1 on NUMA socket 1
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:80:04.2 on NUMA socket 1
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:80:04.3 on NUMA socket 1
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:80:04.4 on NUMA socket 1
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:80:04.5 on NUMA socket 1
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:80:04.6 on NUMA socket 1
EAL:   probe driver: 8086:2021 rawdev_ioat
EAL: PCI device 0000:80:04.7 on NUMA socket 1
EAL:   probe driver: 8086:2021 rawdev_ioat
**Total dev_count: 2**  <------------------------------------------------ FOUND the eth devices
hello from core 1
hello from core 2
hello from core 0
$

onvm_mgr startup output:

./go.sh -k 3 -n 0xF8 -s stdout
INFO: Using default CPU cores 0,1,2

Base virtual address set to default 0x7f000000000
!PORTS! 3
EAL: Detected 24 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: No legacy callbacks, legacy socket not created
total ports: 0 <------------------------------------------------------ rrte_eth_dev_count_avail() returns 0.
WARNING: requested port 0 not present - ignoring
WARNING: requested port 1 not present - ignoring
Registered 5 cores for NFs: 3, 4, 5, 6, 7
Creating mbuf pool 'MProc_pktmbuf_pool' [32767 mbufs] ...
Creating mbuf pool 'NF_INFO_MEMPOOL' ...
Creating mbuf pool 'NF_MSG_MEMPOOL' ...

Checking link statusdone
Default service chain: send to sdn NF
cur_index:1, action:2, destination:1

APP: Finished Process Init.
APP: 3 cores available in total
APP: 1 cores available for handling manager RX queues
APP: 1 cores available for handling TX queues
APP: 1 cores available for handling stats
APP: Core 1: Running TX thread for NFs 1 to 127
APP: Core 0: Running master thread
APP: Stats verbosity level = 1
APP: Core 2: Running RX thread for RX queue 0

PORTS
-----

[twood02]

Unfortunately we do not have mellanox hardware to test this behavior so this is hard for us to debug. When you run the DPDK hello world example are you using the version of DPDK included with ONVM, or a more recent update?

[rudh24]

I use the one included with ONVM. I understand the hardware unavailability. This machine is also equipped with an intel 10G NIC.(this has been bound to dpdk as well). If other people haven't encountered this, this might be a platform specific issue Thank you for your quick response.

[rudh24]

Workaround: Was able to make it work by using an externally installed DPDK. I made sure to include the mellanox pmd in the dpdk config - specifically setting the following in config/common_base

CONFIG_RTE_LIBRTE_MLX5_PMD=y
CONFIG_RTE_LIBRTE_MLX5_DEBUG=y

Can open enhancement to include support for Mellanox. Additionally, since Makefile installation for DPDK is deprecated, and will be removed from 20.11, recommend opening a new enhancement for migrating dpdk install to the meson/ninja based install.