search

rqlite / rqlite

The lightweight, user-friendly, distributed relational database built on SQLite.
https://rqlite.io
MIT License
15.58k stars 708 forks source link

Form clusters via UDP broadcast #293

Closed otoolep closed 9 months ago

otoolep commented 7 years ago

Another idea to ease cluster formation.

Nodes could broadcast IP addresses, via UDP, to 255.255.255.255. This would allow new nodes, when they come up on the same LAN, to automatically detect other rqlite nodes.

A cluster name could be included in the broadcast message, allowing multiple clusters to be formed in this manner on the LAN.

otoolep commented 7 years ago

UDP broadcast doesn't work in AWS, so I am not sure how much this would be used in practise.

mainframe commented 7 years ago

It would be still very useful in private clouds (like Openstack) and in public clouds where UDP broadcasts are allowed.

ghost commented 7 years ago

would it make sense to use QUIC, which is UDP based and 100% golang based ?

otoolep commented 7 years ago

@gedw99 -- can you provide a more precise reference? A URL?

ghost commented 7 years ago

What you can't read minds ?

https://en.m.wikipedia.org/wiki/QUIC

https://github.com/lucas-clemente/quic-go

This one made my day:) https://github.com/simia-tech/netx Read the slide first

riaan53 commented 7 years ago

Discovery can be done nicely with https://github.com/hashicorp/serf :)

adminy commented 2 years ago

UDP broadcast doesn't work in AWS, so I am not sure how much this would be used in practise.

Broadcast no, but there is multicast ;)

Quick implementations now have C++ versions, go versions, definitely worth considering one. If it can speed up the way in which nodes communicate then why not?

kevinmingtarja commented 1 year ago

Yeah as @adminy mentioned, we can do multicast in AWS using Transit Gateways now. So we can use multicast DNS for service discovery and ease cluster formation.

There's an AWS blog post on this: https://aws.amazon.com/blogs/networking-and-content-delivery/automating-service-discovery-using-aws-transit-gateway-multicast-with-igmp/

otoolep commented 9 months ago

rqlite supports a bunch of autoclustering techniques. I don't plan to offer anymore at this time.