In this code, ib_post_send is defined at linux/include/rdma/ib_verbs.h, and the normal kernel RDMA implementations look like support Soft-RoCE. To be more specific, kernel implementation doesn't care what kind of device and device drivers are actually installed, they just work as wrapper functions.
GiantVM looks like using ib_core stack, so theoretically, there's no problem that using GiantVM on Soft-RoCE.
So here is my test plan:
Create two normal QEMU instances.
Connect them with ethernet, and install GiantVM kernel.
In QEMU instance, run GiantVM's QEMU
If Soft-RoCE support is perfect, it works without any problem.
If some problem occur, I'll report it on this issue.
And for other RDMA protocols like iWARP, they depend on TCP/IP, so it's not fit our model. I think those of protocols are for user, not kernel. Cisco report that in iWARP, remote R/W has almost same latency as local memory, but they experiments are based on 100Gbps ethernet card so I think it's not feasible in 10Gbps or 1Gbps ethernet card.
But testing GiantVM on other RDMA protocols sounds fun and interesting, so maybe it can be our next step: Benchmark GiantVM on variance RDMA protocols.
Current RDMA implementation of GiantVM (
linux/arch/x86/kvm/krdma.c
) using a normal RDMA wrapper function that normal linux kernel provided.For example, the
krdma_send
function is implemented like this:In this code,
ib_post_send
is defined atlinux/include/rdma/ib_verbs.h
, and the normal kernel RDMA implementations look like support Soft-RoCE. To be more specific, kernel implementation doesn't care what kind of device and device drivers are actually installed, they just work as wrapper functions.GiantVM looks like using ib_core stack, so theoretically, there's no problem that using GiantVM on Soft-RoCE.
So here is my test plan:
And for other RDMA protocols like iWARP, they depend on TCP/IP, so it's not fit our model. I think those of protocols are for user, not kernel. Cisco report that in iWARP, remote R/W has almost same latency as local memory, but they experiments are based on 100Gbps ethernet card so I think it's not feasible in 10Gbps or 1Gbps ethernet card.
But testing GiantVM on other RDMA protocols sounds fun and interesting, so maybe it can be our next step: Benchmark GiantVM on variance RDMA protocols.
References: