Thoughts for decoupling & abstracting away from the event loop?

[WildCryptoFox]

What I'm proposing is to split the MIO specifics out of the Raft core. Specifically to make sure that the Raft implementation doesn't contain opinionated and possibly consumer blocking constraints such as being limited to Raft owning the event loop, and working only on TCP. Additionally this simplifies the testing suite such that the tests are Raft specific.

• [ ] Depend on the trait interface instead of mio directly;
• [ ] Benchmarks that don't depend on networking and mio allow us to get the true values for the raft implementation;
• [ ] Factor down and split the tests;
  • [ ] Raft tests for protocol;
  • [ ] For tests that require timeout handling, mio may still be required - note: not for networking;
  • [ ] Mio implementation tests for the backend trait and the provided mio:tcp configuration;
• [ ] Let the consumer decide the event_handler:transport configuration that they want (instead of force mio:tcp);
• [ ] Prepare for cryptography preprocessor between capnp and networking with peers.

[Hoverbear]

I like the points you bring up and I think that this is a discussion we should involve @danburkert as well, let's wait to talk about it till he's back?

[WildCryptoFox]

Definitely - already intended to wait for his thoughts. We have already all mentioned and discussed the idea of breaking things into the respective modules a while back on IRC. Thought considering I've implemented it for my own projects; that it should be an issue here for when/if we eventually do abstract the implementation specifics away. :-)

[cramertj]

Has any progress been made on this? I'm interested in contributing if possible. I'm relatively new to Rust, but it seems like an extra hand may be of use here.

[danburkert]

@cramertj not much movement. The library is already split internally into the Consensus and Server types internally. Consensus is responsible for all of the Raft logic, while Server handles sending/receiving messages between peers and clients. The separation is pretty good, but there is probably room for improvements. If you have any ideas they would be appreciated!

[Albibek]

Hi everyone. I've managed to separate the consensus logic of this crate to a separate crate. https://github.com/Albibek/raft-consensus

Feel free to criticise, submit PRs, issues, etc.

[Hoverbear]

Hi @Albibek Awesome! 🎉 :)

Over at @pingcap we also created https://github.com/pingcap/raft-rs with the Raft implementation in production with TiKV. I'm going to be focusing my efforts there.