I have been experimenting lately with rdmacm(1.0.17) library and something strange hapenned.
I am having a single passive side(server) and issuing reads(of 128 MB) from a client to a remote memory. The latency to receive completions for multiple buffers is ok when posting multiple reads. But when adding another queue pair (even if this queue pair have to do with 2 different nic's/2 different physical machines) The latency becomes very high sometimes for some completion events. For a single read request of 128 MB rdma_get_send_comp can take 1 2 or 3 seconds which I guess is not normal.
Some completion events have small delay but at some point the delay becomes orders of magnitude higher.
Is there something I am missing here? Or anything that I might be doing wrong?
Hello,
I have been experimenting lately with rdmacm(1.0.17) library and something strange hapenned. I am having a single passive side(server) and issuing reads(of 128 MB) from a client to a remote memory. The latency to receive completions for multiple buffers is ok when posting multiple reads. But when adding another queue pair (even if this queue pair have to do with 2 different nic's/2 different physical machines) The latency becomes very high sometimes for some completion events. For a single read request of 128 MB rdma_get_send_comp can take 1 2 or 3 seconds which I guess is not normal. Some completion events have small delay but at some point the delay becomes orders of magnitude higher. Is there something I am missing here? Or anything that I might be doing wrong?