Support for RabbitMQ Clusters

[angelokurtis]

Currently, when connecting to RabbitMQ, the client relies on a single host.

var singleHost string = "amqp://guest:guest@localhost"
conn, err := rabbitmq.NewConn(
    singleHost,
    rabbitmq.WithConnectionOptionsLogging,
)

In scenarios where high availability and fault tolerance are crucial, having the ability to specify multiple hosts would allow clients to connect to alternative nodes in case of failures or unavailability.

var hosts = []string{
    "amqp://usr:pwd@127.0.0.1:5672",
    "amqp://usr:pwd@127.0.0.1:5673",
    "amqp://usr:pwd@127.0.0.1:5674",
}
conn, err := rabbitmq.NewClusterConn(
    hosts,
    rabbitmq.WithConnectionOptionsLogging,
)

I'm sure this update will make this library even more awesome for people using it to manage RabbitMQ in production.

Thank you for considering this feature request.

[nemith]

I am wondering if a resolver interface would be good here

Something like

resolver := ClusterResolver([]string{
    "amqp://usr:pwd@127.0.0.1:5672",
    "amqp://usr:pwd@127.0.0.1:5673",
    "amqp://usr:pwd@127.0.0.1:5674",
})

conn, err := rabbitmq.NewClusterConn(
        resolver,
    rabbitmq.WithConnectionOptionsLogging,
)

Where a resolver is an interface:

type Resolver interface {
   Resolve() ([]string, error)
}

The idea behind this is you could support service discovery for clusters this way. For example you could have ConsulResolver or a SRVResolver for looking up hosts via SRV record in DNS.

Only one implementation should be in the library and would be just a ClusterResovlver

Something lile

type ClusterResolver struct {
   uris []string
   random bool
}

Randomize says that each call to Resolve would shuffle the list.

I am happy to contribute here with a PR if it feels like a good direction?

[wagslane]

It doesn't look like the base AMQP library supports clustered connections either: https://github.com/streadway/amqp/issues/339

I'm happy to support this if someone wants to open a PR. I believe the implementation should just be that multiple urls can be specified. The first is always used, but on reconnection it just cycles through the list

[nemith]

Yeah base library doesn't support reconnects either. With clustering you just need to connect to one of the nodes in the cluster. So on reconnect you would try to (potentially) connect to another server.

So it makes sense that the base AMQP library doesn't support it (if you accept their narrow view of what the library should do).

I believe the implementation should just be that multiple urls can be specified. The first is always used, but on reconnection it just cycles through the list

I agree this should be the base functionality but also why i propose the specifying alternative ways of reconnecting. For me a list is no good as the backend systems are going to be dynamic (think kubernetes) and needs to be accessed via service discovery.

This is similar to the HostProvider in the go-zookeeper/zk although much simpler. I think the "hostprovider" or whatever it should me named just always return a list to cycle through on each reconnect.

I do think that a single implementation of this should be in this library with other implementations could be provided by the caller unique to their environment.