danthegoodman1 / raftd

Multi-group Raft as a daemon. Build a custom bulletproof distributed storage service in any language on top of high performance multi-group Raft just by implementing a handful of HTTP endpoints.
3 stars 0 forks source link

node join support #13

Closed danthegoodman1 closed 1 week ago

danthegoodman1 commented 3 weeks ago

The issue is that node join support is not super elegant https://pkg.go.dev/github.com/lni/dragonboat/v4#NodeHost.StartReplica

  • starting a brand new Raft shard, set join to false and specify all initial member node details in the initialMembers map.
  • joining a new node to an existing Raft shard, set join to true and leave the initialMembers map empty. This requires the joining node to have already been added as a member node of the Raft shard.
  • restarting a crashed or stopped node, set join to false and leave the initialMembers map to be empty. This applies to both initial member nodes and those joined later.

This means we have to know between an initial join, and a restart. We also need to know if it's an initial member or not. It seems that the initial member is not damaging when recovering: https://github.com/lni/dragonboat-example/blob/master/helloworld/main.go#L128C2-L132C25

// when joining a new node which is not an initial members, the initialMembers // map should be empty. // when restarting a node that is not a member of the initial nodes, you can // leave the initialMembers to be empty. we still populate the initialMembers // here for simplicity.

The reason is that dragonboat "recruits" members my reaching out to them and telling them to join, so they need to understand that they are in the join phase. In order to handle joining, it would be nice if we can detect that this is not an initial member, and if there's local state of already being in a cluster.

danthegoodman1 commented 1 week ago

What we need to do is override the initialMembers to nothing if join is provided, and non-initial nodes should always leave join provided

danthegoodman1 commented 1 week ago

Or even better, we just detect whether it's an initial node or not based on it's URL and the initialMembers

danthegoodman1 commented 1 week ago

Initial cluster will bootstrap with shard 0

danthegoodman1 commented 1 week ago

kinds of depends on shard creation, but mostly done