hashicorp / memberlist

Golang package for gossip based membership and failure detection
Mozilla Public License 2.0
3.62k stars 435 forks source link

Not able to perform List.Members() inside a NotifyJoin()/NotifyLeave() as it results in deadlock #237

Open sandyydk opened 3 years ago

sandyydk commented 3 years ago

Not able to perform List.Members() inside a NotifyJoin()/NotifyLeave() as it results in deadlock. I see a single lock being used for the both. is there a way to find total nodes in the cluster when an event occurs?

sandyydk commented 3 years ago
func (m *Memberlist) Members() []*Node {
    m.nodeLock.RLock()
    defer m.nodeLock.RUnlock()
func (m *Memberlist) aliveNode(a *alive, notify chan struct{}, bootstrap bool) {
    m.nodeLock.Lock()
    defer m.nodeLock.Unlock()

Both end up trying to acquire the same lock.

ozon2 commented 3 years ago

I had the same issue. I ended up using a channel to avoid using list.Members() inside NotifyJoin().

type eventDelegate struct {
    eventChan chan memberlist.NodeEventType
}

func (ed *eventDelegate) NotifyJoin(node *memberlist.Node) {
    ed.eventChan <- memberlist.NodeJoin
}
for {
    event := <-eventChan
    members := list.Members()
}