Compiling Error on cluster with no DPDK library

[JingyZhu]

I'm trying to run your eRPC on the CloudLab cluster. After I run:

 cmake . -DPERF=OFF -DTRANSPORT=infiniband
 make

I got an error message:

In file included from /users/jingzy/eRPC/src/cc/timing_wheel.h:19:0,
                 from /users/jingzy/eRPC/src/session.h:8,
                 from /users/jingzy/eRPC/src/nexus.h:6,
                 from /users/jingzy/eRPC/src/nexus_impl/nexus.cc:1:
/users/jingzy/eRPC/src/transport_impl/dpdk/dpdk_transport.h:12:10: fatal error: rte_common.h: No such file or directory
 #include <rte_common.h>

I guess it might be the reason that my cluster doesn't install DPDK's library. But since I'm running on the InfiniBand mode, is there any way to fix this without installing that library?

By the way, I notice that an autorun_app_file should be used to indicate the app that needs to be compiled together with. In you CMakeList, the .cc should have the same name as its directory. How can I do this for app/hello since I find no hello.cc inside hello?

[anujkaliaiitd]

I've pushed fixes for these issues. To avoid confusion, the hello application is now outside apps.

[JingyZhu]

I just pulled the repository and remake the source code. There seems to be some new error:

In file included from /users/jingzy/eRPC/src/transport_impl/raw/raw_transport.h:7:0,
                 from /users/jingzy/eRPC/src/cc/timing_wheel.h:21,
                 from /users/jingzy/eRPC/src/session.h:8,
                 from /users/jingzy/eRPC/src/nexus.h:6,
                 from /users/jingzy/eRPC/src/nexus_impl/nexus.cc:1:
/users/jingzy/eRPC/src/transport_impl/raw/mlx5_defs.h:8:10: fatal error: infiniband/verbs_exp.h: No such file or directory
 #include <infiniband/verbs_exp.h>

I also find some other files that include this header, but I only see verbs.h not verbs_exp.h in /usr/include/infiniband. Is there any way to install that header?

[anujkaliaiitd]

Try installing the latest Mellanox OFED drivers.

[JingyZhu]

I install the latest Mellanox OFED driver possible supports my OS (Ubuntu 14.04.1 LTS, Driver: 4.2-1.2.0.0). And I succeeded in compiling the eRPC. But when I run sudo ctest, almost every test ended with

      Start  1: create_session_test
 1/14 Test  #1: create_session_test ..............***Exception: Child aborted  0.04 sec

I went to hello_world and ran server, and I got the error message:

87:501384 WARNG: eRPC Nexus: Testing enabled. Perf will be low.
libibverbs: Warning: couldn't load driver '/usr/lib/libibverbs/libipathverbs': /usr/lib/libibverbs/libipathverbs-rdmav2.so: cannot open shared object file: No such file or directory
87:506338 WARNG: Modded driver unavailable. Performance will be low.
87:506459 WARNG: eRPC HugeAlloc: Insufficient memory. Can't reserve 16 MB.
terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to allocate 15MB for ring buffers.

I'm really not familiar with this. What could be the reason?

[anujkaliaiitd]

libipathverbs is the driver for QLogic NICs, which are currently not supported in eRPC.

Mellanox InfiniBand NICs are available in CloudLab's Apt cluster.

[JingyZhu]

This is weird..... Because I'm actually running eRPC on the CloudLab's APT cluster. I typed lspci | grep Network and got:

08:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]

And in the directory /usr/lib/libibverbs/ which reports the error, there are two files libmlx4-rdmav2.so libmlx5-rdmav2.so I just follow the step on README using InfiniBand mode and make the library.

[anujkaliaiitd]

Thanks for clarifying. Have you ensured "Unlimited SHM limits, and at least 512 huge pages on every NUMA node." See https://github.com/efficient/mica2/blob/master/script/setup.sh for an example of how to do so.

[JingyZhu]

It works! Thanks a lot!