Open Hoverbear opened 9 years ago
Note that membership changes are simpler in the dissertation, but don't forget about this minor gotcha: https://groups.google.com/forum/#!topic/raft-dev/t4xj6dJTP6E
@ongardie Would you suggest using one (the one in the paper) over the other (the one in your dissertation?)
I'd suggest the single-server approach (dissertation), unless you have a good reason otherwise. (LogCabin still uses the joint consensus approach but just because I never went back to update the code.)
@ongardie I was reviewing this all today (made some notes above) and was wondering if you've found other applications for the Leader Transfer extension? The other suggested methods of removing the current leader seem rather complicated.
Nice summary, @hoverbear, but don't forget to add the fix for the bug I linked to in my first comment!
I don't think I've learned more uses for leadership transfer in the last year since my dissertation was published, so my thoughts in 3.10 are still current. Overall, I think it'd be cleaner and more useful to implement the leadership transfer approach and then use it when removing the cluster leader. But I also have to admit I still have never implemented leadership transfer myself (maybe others on raft-dev have?).
I think it's also worth outlining adding a member from the perspective of the new member. A Raft instance must respond to anyone trying to talk to it, and there must be a way to "boot" it without it turning into a one-group node. Instead it should just sit there and wait for incoming messages, in case it is added to an existing group. This "talking to strangers" is new with config changes and at the heart of why they're probably the most complex and error-prone part of Raft.
Support dynamic membership changes as specified in the dissertation. (See
README.md
) This is a simpler choice.Notes from the dissertation:
Safety
Leader
should only respond to the client when the majority have commited the configuration change.AppendEntries
requests from a leader that is not part of the server's latest configuration. This is because it may not yet have the entry where the server is added.RequestVote
, this may occasionally be needed to keep the cluster available.Availability
Leader
must determine when the new server is sufficiently caught up to continue. "Round based" catchup as detailed in page 38 of dissertation in the last paragraph is a good way of doing this.Leader
must abort the change if it's too slow or unavailable. (This risks disrupting the cluster)SocketAddr
to test for failure.next_index
counter finally drops to 1 and the log starts replicating. It is suggested that theFollower
s include the length of their logs in theAppendEntries
response this way theLeader
can cap it.Removing the Current Leader
Distruptive Servers
RequestVote
RPC is modified such that:RequestVote
request within the minimum election timeout of hearing from a current leader it does not update it's term or grant a vote. Dropping, replying invalid, or delaying the response is fine.RequestVote
requests under such a condition.