zuiderkwast
commented
1 month ago RDMA is the abbreviation of remote direct memory access. It is a technology that enables computers in a network to exchange data in the main memory without involving the processor, cache, or operating system of either computer. This means RDMA has a better performance than TCP, the test results show Valkey Over RDMA has a ~2.5X QPS and lower latency.
In recent years, RDMA gets popular in the data center, especially RoCE(RDMA over Converged Ethernet) architecture has been widely used. Cloud Vendors also start to support RDMA instance in order to accelerate networking performance. End-user would enjoy the improvement easily.
Introduce Valkey Over RDMA protocol as a new transport for Valkey. For now, we defined 4 commands:
GetServerFeature & SetClientFeature: the two commands are used to negotiate features for further extension. There is no feature definition in this version. Flow control and multi-buffer may be supported in the future, this needs feature negotiation.
Keepalive
RegisterXferMemory: the heart to transfer the real payload.
The 'TX buffer' and 'RX buffer' are designed by RDMA remote memory with RDMA write/write with imm, it's similar to several mechanisms introduced by papers(but not same):
Socksdirect: datacenter sockets can be fast and compatible https://dl.acm.org/doi/10.1145/3341302.3342071
LITE Kernel RDMA Support for Datacenter Applications https://dl.acm.org/doi/abs/10.1145/3132747.3132762
FaRM: Fast Remote Memory https://www.usenix.org/system/files/conference/nsdi14/nsdi14-paper-dragojevic.pdf
This text is useful to add in the documentation, at least some of it, and the links?
When this is merged, users will not find the description of the PR. :-) I think the PR description can just be "Add documentation for the RDMA feature added in https://github.com/valkey-io/valkey/pull/477"
pizhenwei
RDMA is the abbreviation of remote direct memory access. It is a technology that enables computers in a network to exchange data in the main memory without involving the processor, cache, or operating system of either computer. This means RDMA has a better performance than TCP, the test results show Valkey Over RDMA has a ~2.5X QPS and lower latency.
In recent years, RDMA gets popular in the data center, especially RoCE(RDMA over Converged Ethernet) architecture has been widely used. Cloud Vendors also start to support RDMA instance in order to accelerate networking performance. End-user would enjoy the improvement easily.
Introduce Valkey Over RDMA protocol as a new transport for Valkey. For now, we defined 4 commands:
The 'TX buffer' and 'RX buffer' are designed by RDMA remote memory with RDMA write/write with imm, it's similar to several mechanisms introduced by papers(but not same):