search

danthegoodman1 / raftd

Multi-group Raft as a daemon. Build a custom bulletproof distributed storage service in any language on top of high performance multi-group Raft just by implementing a handful of HTTP endpoints.
3 stars 0 forks source link

shard creation support (and rehaul raft group management) #17

Open danthegoodman1 opened 1 week ago

danthegoodman1 commented 1 week ago

Explicitly create a shard, and have that node immediately become single raft node leader.

Maybe we can expect that the initial members will be the same and should all get the shard added? Then this has to be called on all of the initial nodes roughly around the same time, and they will bootstrap the initial cluster

We're already managing storing the known shards for the cluster, it's simplified if we make the initial members always the same (assuming they're never truly "lost", as it's just DNS)

danthegoodman1 commented 1 week ago

Initial members vs other nodes will probably have to be handled differently, and then we need to consider initial members for this specific group as well (set the initial members for a given shard).

danthegoodman1 commented 1 week ago

Right now the problem is we're straddling cockroachdb-like usage (just specify one set of initial members) and lower-level control where we let the user define groups. We might just need to never start an initial group and have them call down to the new to specify each cluster (stored as a JSON file in the dir?)