Is it possible to run a simulated ePRC?

[shenweihai1]

let me elaborate on the question. Suppose that

• I can't succeed in deploying DPDK/RDMA on the server due to NICs or whatever issues
• I can succeed in compiling ePRC

In this setting, I do not want to block my project and code my project as soon as possible (I can solve the DPDK/RDMA deployment issue later).

To achieve this, I need to have the simulated ePRC which shares the exact same interfaces as the normal ePRC so that I do not need to modify my code later when DPDK/RDMA is ready. The performance of simulated ePRC does not matter, it can go through CPU and network stack as the normal TCP/IP network because I only use this for my development and to verify the correctness of my code.

So, is there any way to support this feature?

[anujkaliaiitd]

Hi Weihai. One way to do this is to create a SoftRoCE device (an emulated RoCE NIC that supports the ibverbs API), and build eRPC with -DTRANSPORT=infiniband -DROCE=on. I've got this to work several years ago for the exact purpose that you mention.

[shenweihai1]

It works! Thank you so much! In case someone else needs this, I kept a note for steps on Ubuntu 20.04 (it does not work on Ubuntu18.04)

# 0. prerequisites
sudo apt install libibverbs-dev librdmacm-dev 
sudo apt install build-essential cmake gcc libudev-dev libnl-3-dev libnl-route-3-dev 
sudo apt install ninja-build pkg-config valgrind python3-dev cython3 python3-docutils pandoc
sudo apt install rdmacm-utils

# 1. install rdma-core
sudo apt update
git clone https://github.com/linux-rdma/rdma-core
apt-get --assume-yes install build-essential cmake gcc libudev-dev libnl-3-dev libnl-route-3-dev ninja-build pkg-config valgrind python3-dev cython3 python3-docutils pandoc
cd rdma-core
bash build.sh
cmake .
sudo make -j10
sudo make install

# 2. add a device (have to re-create it once reboot)
# get netdev by dmesg | grep -i ethernet
# rdma link is new feature on Ubuntu20.04
sudo rdma link add r0 type rxe netdev eno1
# rdma link
# rdma dev
# rdma resource
# ibv_devices 
# ibv_devinfo
# ibstat

# 3. testing
## server
rping -s -v
## client
rping -c -v -a  server-ip

# 4. install eRPC
cd ~
git clone https://github.com/erpc-io/eRPC.git
cd eRPC
# remove "-Werror" and add set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-address-of-packed-member") in CMakeLists.txt
cmake . -DTRANSPORT=infiniband -DROCE=on
make -j10

# 5. test: modfiy the ip and port (using different port for client and server)
## server
sudo ./build/hello_server
## client
sudo ./build/hello_client