search

lni / dragonboat

A feature complete and high performance multi-group Raft library in Go.
Apache License 2.0
5.08k stars 541 forks source link

custom transport and memberlist support #154

Closed aep closed 3 years ago

aep commented 3 years ago

Would be great if we could use this library on top of an existing encrypted mesh.

https://github.com/lni/dragonboat/blob/ef3c05980fad88e99e0b079445dbff9985828f61/config/config.go#L51

looks like there is already an api for it, but its not documented and fairly complex.

we already have a list of members, and a (message or stream based) transport method to them through devguard carrier, but just need consenus on top. Is that something that would fit into dragonboat?

lni commented 3 years ago

@aep That TransportModule field was recently added, it is still a work in progress feature that only exists in the development branch. TestTransportModuleCanUseNodeHostID in nodehost_test.go is an example on how to use it, you may want to have a look.

I am working on this feature as in my own project I'd like to use a gossip protocol (e.g. similar to hashicorp's memberlist library) to maintain/discover/query all NodeHost instances. Rather than addressing each NodeHost using its IP:port or DNS name:port (specified as the RaftAddress value in NodeHostConfig), I'd like to address NodeHost instances by their NodeHostID values which are uniquely assigned to each NodeHost and will stay unchanged. A custom TransportModule will be used in my project to implement the above described idea.

For your project, if all your machines in the mesh can guarantee to have stable DNS names that won't change, maybe you just need to take over the transport part, in that case github.com/lni/dragonboat/v3/plugin/chan might a useful example to look into.

lni commented 3 years ago

Some changes have been made in the last few weeks.

NodeHostConfig.Expert.TransportFactory can be set to use custom transport module. plugin/chan/chan.go is an example on how to implement such a custom transport module.

NodeHostConfig.AddressByNodeHostID can be set to request all NodeHost instances to be addressed by an internal randomly assigned static ID value. This allows NodeHost instances to use dynamic NodeHostConfig.RaftAddress values, e.g. when your machines have dynamically assigned IPs on each reboot. The implementation uses hashicorp's memberlist package to exchange NodeHostID to RaftAddress mapping info by gossip to allow NodeHostID to be transparently translated to its latest known RaftAddress value when necessary. TestGossip in nodehost_test.go shows how to use this new gossip feature.

NodeHostConfig.AddressByNodeHostID and NodeHostConfig.Expert.TransportFactory can be used together. See TestCustomTransportCanUseNodeHostID in nodehost_test.go as an example.