Add protocol buffers support for messages and gRPC services

[sainad2222]

Problem statement Currently, Openraft does not use Protocol Buffers (proto) messages; instead, it passes Rust structs for internal communication. This necessitates that anyone writing a gRPC network layer must rewrite proto files like the example found in databend's source code

Solution Openraft can have proto file and generated pb file in the repo along with From and Into trait converters that people can use in writing thier network component Example from kv-memstore: Instead of

#[post("/add-learner")]
pub async fn add_learner(app: Data<App>, req: Json<(NodeId, String)>) -> actix_web::Result<impl Responder> {
    let node_id = req.0 .0;
    let node = BasicNode { addr: req.0 .1.clone() };
    let res = app.raft.add_learner(node_id, node, true).await;
    Ok(Json(res))
}

it will look something like this

#[post("/add-learner")]
pub async fn add_learner(app: Data<App>, req: add_learner_request) -> add_learner_response {
    let node_id = req.node_id;
    let node = BasicNode { addr: req.addr.clone() };
    let res = app.raft.add_learner(node_id, node, true).await;
    Ok(res.into())

Scope and tools: The plan is to utilize prost and tonic for handling proto files and generating Rust code. As I delve deeper into the codebase, I will update this section with more details.

Open questions:

1. Should protobuf generator be pluggable? Ex: rust-protobuf instead of prost

[github-actions[bot]]

👋 Thanks for opening this issue!

Get help or engage by:

• /help : to print help messages.
• /assignme : to assign this issue to you.

[sainad2222]

/assignme

[schreter]

The plan is to utilize prost and tonic for handling proto files and generating Rust code.

Though I understand that some people might want to use gRPC, this is way too slow/heavyweight for our use case. Further, the mentioned tools currently pretty much hard-code tokio runtime, to which we painstakingly removed the dependencies from the core.

Please ensure while doing so to put such things into a separate crate not polluting the openraft core itself. It is not a core functionality. But, I assume, this was your plan from the beginning, right? :-)