/dev/dpdk-iface does not show up

[tbarbette]

Hi,

I'm trying to launch epserver but I get "Error opening dpdk-face".

Indeed dpdk-iface.ko is loaded but there is no /dev/dpdk-iface entry. I use your dpdk version from git submodule.

I have a mlx5 NIC. Kernel 4.15. Ubuntu 18.04.

epserver.conf: io = dpdk num_mem_ch = 6 port = enp115s0f0 rcvbuf = 8192 sndbuf = 8192 tcp_timeout = 30 tcp_timewait = 0 stat_print = dpdk0

Thanks, Tom

[conjam]

I am having a similar issue, and I am also running dpdk with an mlx5 NIC.

You can get around this by creating the device if you run the following.

cd mtcp/dpdk-iface-kmod/
insmod dpdk_iface.ko
sudo ./dpdk_iface_main 

As a meta point, this kernel module for your set up (and mine) seems unneeded as it is only used for Intel NICs.

As a side note, this kernel module will be automatically inserted if you answer 'y' to using an Intel NIC when you run setup_mtcp_dpdk_env.sh.

[tbarbette]

Thanks for the help @conjam

Yes it's probably not needed, but epserver and lighttpd refuse to start if that module is not there... I get this error with that script :

nslrack14 [583] % sudo ./dpdk_iface_main
Removing existing device node entry... not present.
Creating device node entry... done.
Setting permissions on the device node entry... done.
Scanning the system for dpdk-compatible devices...EAL: FATAL: Cannot get hugepage information.
No Ethernet port detected!

EDIT : Ok that second specific issue was because mTCP only supports 2MB hugepages apparently (or that module/the tool it calls?)

I'll leave the issue open until some authors pass by, the overall experience/doc could be increased. Especially if that module is indeed not needed.

[tbarbette]

@conjam did you manage your mlx5 NIC to be detected? epserver does not seem to launch the mlx5 discovery and I get no ethernet ports, but testpmd does... Strange.

[conjam]

@tbarbette Oops, sorry for the delay.

The core of the problem here is when probe_all_rte_devices() is called (io_module.c, line 314), it asks the dpdk-iface kernel module what the PCI address is for your NIC. From my limited perspective, it seems this function is how mTCP automatically populates arguments for rte_eal_init().

For my setup, if I had my interface correctly specified in my conf file, then the iface kernel module would simply populate argv with PCI addresses to devices that do not exist.

Thus when rte_eal_init() is invoked later, it will try to talk to PCI addresses that don't exist, and you'll get the dreaded "no ethernet port" message. My band-aid was to hardcode my PCI addresses after the -w flag in the argv arguments.

NOTE: If the argv inputs look correct, try running testpmd with those arguments; it is possible that there could be a problem with dpdk.

[ajamshed]

@tbarbette ,

Apologies for the mess-up. I hadn't tested mTCP with mlx4/mlx5 devices in a long time. I have pushed some code changes in the devel branch. These changes should, at least, get mTCP working with mlx5 NICs. Please let me know if you still observe some unresolved issues. I plan to clean up the code and synch up all the changes from devel branch to the master branch in the coming days.

[tbarbette]

Yes it is solved with devel, thanks !

[tbarbette]

However, apache benchmark (and I would guess other clients) do not work with multiple cores. Certainly the RSS key issue as with mOS.

[ajamshed]

@tbarbette ,

We need to reverse engineer the rss algorithm for Mellanox NIC as well. Thanks for the reminder. I will try to find some time to work on this.

[WenqingWu]

@tbarbette @ajamshed I got mtcp-devel code, and tried to run epserver with a mlx5 NIC(ConnectX-4 Lx) on an ARM server: $ sudo ./epserver -p /home/xxx/www -f epserver.conf -N 2

And I got error:

Loading mtcp configuration from : epserver.conf Loading interface setting EAL: Detected 40 lcore(s) EAL: Detected 1 NUMA nodes EAL: Auto-detected process type: PRIMARY EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: No free hugepages reported in hugepages-2048kB EAL: Probing VFIO support... [ SetNetEnv: 356] No Ethernet port!

Linux: CentOS Linux release 7.6.1810 (AltArch) Kernel:4.14.0-115.el7a.0.1.aarch64 epserver.conf: io = dpdk num_mem_ch = 4 port = eth1 rcvbuf = 8192 sndbuf = 8192 tcp_timeout = 30 tcp_timewait = 0 stat_print = eth1

@tbarbette How did you get your problem solved? Thanks, WenqingWu

[WenqingWu]

It works now。

I modified mtcp/src/io_module.c: comment code in function "probe_all_rte_devices". But I am not quite sure about the reason.

[laiyongGit]

Hi, I encountered this problem while running epserver

./epserver -p ./ -f epserver.conf -N 2

Configuration updated by mtcp_setconf(). Loading mtcp configuration from : epserver.conf Loading interface setting [probe_all_rte_devices: 122] Error opening dpdk-face!

[ajamshed]

Apologies for the delayed response. Please make sure that dpdk_iface.ko driver is correctly inserted. If you have any follow-up questions please reply. I will try to respond back more promptly.

[nilanjana87]

Hi, dpdk_iface.ko is correctly inserted in my system. I am using Mellanox ConnectX-5 NICS & linux version 4.15.0-96-generic. I am also using the devel branch code. Testpmd succeeds, but epserver gives me the error "[ SetNetEnv: 360] No Ethernet port!"

Any idea how to fix this?

[Naibaowjk]

insmod dpdk_iface.ko

It works now。

I modified mtcp/src/io_module.c: comment code in function "probe_all_rte_devices". But I am not quite sure about the reason.

Hello, Could you tell us what did you done to comment ? just comment all or something else?

Thank you at first :_)

[yasuhironet]

Inspired by the WenqingWu's hint, it started working (partially) for me, too. I think WenqingWu meant that he commented out the below line from the dpdk-iface-kmod/dpdk_iface_main.c (in main()), and then the "dpdk-iface-main" program started to work.

-       ret = probe_all_rte_devices(rte_argv, &rte_argc);
+       //ret = probe_all_rte_devices(rte_argv, &rte_argc);