Service discovery

[RangerMauve]

Currently Docker Swarm provides a great way to load balance a service seamlessly. However in some cases you want the individual service instances to be replicated, and that means they need to somehow find each other.

Going over how one would do that would probably very useful for deploying replicated databases and the such.

[daneroo]

Not sure what you mean by: individual service instances to be replicated... I'm assuming you don't just mean something like scale>1.

[RangerMauve]

Say you have a service for a database, like Redis. And when you scale up your service, you want all the instances to replicate with each other. To join a replica set, the new server instances will usually need to be aware of each other to initiate connections. It'd be useful to see if there was a standard or built in way of getting them to talk to each other.

[hairyhenderson]

Good idea - it's easy to get the processes to scale, but clustered services don't auto-discover each other so there's usually some magic involved in how this works.

A coworker of mine recently started going through this for RabbitMQ. One of the big keys here is a good entrypoint script to configure stuff properly (so #2 might be a good lead-in to this?).

I'll ask my coworker if he's interested in giving a talk on this at some point :wink:

[daneroo]

thanks @RangerMauve. I like this too.

[RangerMauve]

Potentially I'll have figured this out some time in the next couple of weeks. So if that's the case I might try going at it. Though so far my solution doesn't sound like it'll be totally trivial.

[hairyhenderson]

@RangerMauve - that'd be awesome! :grin:

[EtienneDufresne]

@RangerMauve @hairyhenderson At work, I had to do POC of a dockerized rabbitmq cluster where nodes needed to join the cluster on startup. This POC did not use docker swarm but I would be willing to create an open source version of this POC with docker swarm support and present it at the August meetup.

[RangerMauve]

@EtienneDufresne That would be pretty cool. Did you use one of those fancy key-value stores for getting them to talk to each other? I was thinking of using a centralized event broker.

[EtienneDufresne]

At Qlik, we use Consul for service registration and discovery.

https://www.consul.io/ https://github.com/gliderlabs/docker-consul https://github.com/gliderlabs/registrator

[daneroo]

That's a great presentation idea @RangerMauve!

[hairyhenderson]

/cc @DockerOttawaMeetup/organizers - shall we put this on the tentative schedule for the August meetup?

[hairyhenderson]

Many thanks to @EtienneDufresne for delivering this talk! Slides are up: DockerOttawaMeetup/Slides#6

[RangerMauve]

Shame I missed the talk, but the slides are super useful Thanks!!

[EtienneDufresne]

👍