Mellanox / libvma

Linux user space library for network socket acceleration based on RDMA compatible network adaptors
https://www.mellanox.com/products/software/accelerator-software/vma?mtag=vma
Other
557 stars 152 forks source link

NOT see any rocev2 packet #1036

Closed forsakening closed 11 months ago

forsakening commented 12 months ago

Hi all,i run redis-benchmark with libvma,and expect the communication of redis using rocev2,here is my command: Client: D_PRELOAD=/usr/lib64/libvma.so redis-benchmark -h 10.39.30.220 -p 6379 -c 1 -n 1000000 -t get Server:LD_PRELOAD=/usr/lib64/libvma.so redis-server /etc/redis.conf My HCA is mlx5,and show_gids,here is the output: mlx5_2 1 3 0000:0000:0000:0000:0000:ffff:0a27:1ee7 10.39.30.231 v2 ens8

the redis communication is ok, but i can not see any rocev2 packets,seem like the libvma use something like cx5 blueflame,and transport with ethernet packet。

igor-ivanov commented 11 months ago

Hello @forsakening, Could you try following way as https://hub.docker.com/r/mellanox/tcpdump-rdma

igor-ivanov commented 11 months ago

@forsakening can the issue be closed?

forsakening commented 11 months ago

@igor-ivanov hi,i am using tcpdump-rdma,but the capture packet is all raw tcp packet,not rocev2 packet

igor-ivanov commented 11 months ago

so redis uses socket tcp connections and you see tcp packets.

forsakening commented 11 months ago

@igor-ivanov ahhh,so libvma only use rdma verbs,but the underlay packet is raw Ethernet packet, not the rocev2 udp packet ?

igor-ivanov commented 11 months ago

libvma accelerates socket based application. See left branch on the picture.

image

forsakening commented 11 months ago

@igor-ivanov yeah i know libvma call the rdma verbs ,but my test is under cx5 nic,and the packet is raw ethernet,not the rocev2 packet . in my opinion, libvma call rdma verbs , the packet shoule be rocev2 (under cx5).

igor-ivanov commented 11 months ago

no, vma forms tcp/udp packets and send them

forsakening commented 11 months ago

@igor-ivanov thank you very much