hashicorp / memberlist

Golang package for gossip based membership and failure detection
Mozilla Public License 2.0
3.61k stars 435 forks source link

Too many pending push/pull requests #259

Open nikfot opened 2 years ago

nikfot commented 2 years ago

Hello,

I am using memberlist as part of a caching architecture. In the delegate interface, I update local outdated or empty cache in MergeRemoteState with a fresh cache instance from other members in LocalState. According to the documentation I have found this is the way to go, please correct me if I misunderstood something.

For some reason I have not yet been able to identify 100%, after some time I get the error Too many pending push/pull requests and that member practically becomes unresponsive.

I can see that this error is produced in net.go of the pkg from open push pull requests. However I cannnot find a way to clean up those push pull requests nor the reason they are accumulated.

I checked the Broadcast queue and it is empty at any time. So I am not sure if I implementing something the wrong way or if I miss sth or both.

Could you provide some extra insight please?