search

emmericp / MoonGen

MoonGen is a fully scriptable high-speed packet generator built on DPDK and LuaJIT. It can saturate a 10 Gbit/s connection with 64 byte packets on a single CPU core while executing user-provided Lua scripts for each packet. Multi-core support allows for even higher rates. It also features precise and accurate timestamping and rate control.
MIT License
1.04k stars 234 forks source link

Compilation issues with Mellanox NIC (./build.sh --mlx4) #256

Closed 108anup closed 4 years ago

108anup commented 4 years ago

I get the following error when I run ./build.sh --mlx4: /usr/bin/ld: cannot find -lrte_pmd_mlx4 Do I need to compile / install dpdk manually? Or do I need to export some variables? (I saw in the dpdk config in libmoon/deps/dpdk/config that it is not compiling as shared libraries, so I don't this this would be an issue.)

I have installed OFED 4.4 and am on firmware 2.42 Mellanox Connect X3 (OEM - Dell)

108anup commented 4 years ago

I also get many errors like this in the compilation output, I am not sure if this is due to the unavailability of the library or this causes the library to not compile:

MoonGen/libmoon/deps/dpdk/drivers/net/mlx4/mlx4_flow.c:829:7: error: implicit declaration of function ‘ibv_exp_create_cq’ [-Werror=implicit-function-declaration] cq = ibv_exp_create_cq(priv->ctx, 1, NULL, NULL, 0,

108anup commented 4 years ago

BTW: Using Ubuntu 16.04.6 with kernel: 4.4.0-142-generic

108anup commented 4 years ago

And scrolling further up in the build output, I am also getting these errors: MoonGen/libmoon/src/memory.c:2:22: fatal error: rte_mbuf.h: No such file or directory compilation terminated. MoonGen/libmoon/src/device.c:2:25: fatal error: rte_ethdev.h: No such file or directory compilation terminated.

108anup commented 4 years ago

Stumbled upon similar issue: https://github.com/emmericp/MoonGen/issues/169 , the solution says to disable KNI mod CONFIG_RTE_KNI_KMOD=n but I checked and it is already disabled.

emmericp commented 4 years ago

please post the first error message you get while compiling DPDK

108anup commented 4 years ago

the first error message is: /users/anup/MoonGen/libmoon/deps/dpdk/drivers/net/mlx4/mlx4.c: In function ‘txq_cleanup’: /users/anup/MoonGen/libmoon/deps/dpdk/drivers/net/mlx4/mlx4.c:1027:37: error: storage size of ‘params’ isn’t known struct ibv_exp_release_intf_params params;

I don't know if this helps.

emmericp commented 4 years ago

looks like either a header is missing or the wrong version for the mlx4 driver, try a version that is compatible with this DPDK version, see our readme-mlx file for versions that we've tested

108anup commented 4 years ago

The headers are there in: MoonGen/libmoon/deps/dpdk/lib/librte_ether and I am using mlx4 driver version 4.4 which has been tested according to https://github.com/libmoon/libmoon/blob/master/install-mlx.md . I can try again with mlx4 version 4.1, but I am not sure if that will solve the issue.

108anup commented 4 years ago

Okay, so I manually installed the .debs mentioned in the README rather than installing whole OFED driver. 

  libibverbs1[...].deb
  libibverbs1-dev[...].deb
  <OPTIONAL> libibverbs1-dbg[...].deb

  libmlx5[...].deb
  libmlx5-dev[...].deb
  <OPTIONAL> libmlx5-dbg[...].deb

  mlnx-ofed-kernel-only[...].deb
  mlnx-ofed-kernel-dkms[...].deb
  mlnx-ofed-kernel-utils[...].deb

Despite having installed the full driver (OFED 4.4), installing the .deb made some version downgrades (I never installed a higher driver version though). After this I am not getting the above mentioned errors. Thanks!