danthegoodman1 / UltraQueue

Distributed pull queue based on Segment's Centrifuge. Zero dependency p2p clustering.
GNU Affero General Public License v3.0
1 stars 0 forks source link

Use etcd instead of gossip for service discovery? #50

Open danthegoodman1 opened 2 years ago

danthegoodman1 commented 2 years ago

Related to #35

If we use etcd instead for service discovery (and for locking!) then we can achive more nodes without complex gossip work.

Nodes could more aggresively update and query remote partition lengths at scale.

However the etcd cluster would have to be multi-region.

Now we need to involve etcd, and at small scale the benefits are probably not there.