Highly Available Socket Application!

[yatki]

As an example, I'd like to see how you build a HA chat/messaging application as a cluster in a docker swarm (3 nodes maybe) using socket.io-redis.

When a user sends a message, message should be stored to a MongoDB, redis client should publish a message to subscribers and when subscriber nodes get the message event they can emit the message to users in realtime.

I'm interested in docker swarm and redis but if you prefer better tools it's fine. I just would like to see how you would approach to solution.

Thanks :)

[yamalight]

OK, I get the idea of messaging app that has scalable architecture, but why do you want to see swarm in here as well? Just for the sake of it? 😂

[yatki]

@yamalight Yes :D Would be cool i guess :) Because handling load balancing and networking become more challenging with docker in swarm mode.

But, I'd like to see how would you handle scaling without swarm too :D

Alles gut for me.

[yamalight]

Horizontal scalability can be demonstrated without swarm pretty easily - just add more instances behind load balancer. All the swarm does is allows you to run across several different servers. There's actually not that much difference in networking and load balancing between swarm/non-swarm.

[yatki]

I think it's a bit trickier, brings a bit pain or simply I don't know the easy way to do it.

Because container IPs in swarm are assigned dynamically. When i wanted to do reverse proxy configuration I passed service name as the hostname, and then load balancing was being handled by swarm. One trickier part is nginx is not starting before containers created because it can't resolve the hostname. So i had to use wait-for-it.sh script for instance.

I made it work in my way, and it works fine. The reason i suggested this topic is I think it could be interesting for other people too and I'm curious about if there could be another, better approach to do it.

Having 3 nodes on the other hand is completely unnecessary. I thought maybe could be cool to show it as well. But as long as we have several replicas of the service it's fine to have single node.

[yamalight]

Nginx is simply not the best pick for putting in front of the swarm 😄

Alright, to sum it up:

1. Build a messaging app that supports horizontal scalability (e.g. using redis)
2. Deploy it to swarm (one node should be enough i think?)
3. Show how horizontal scalability works

That's it, right?

[yatki]

Sounds great already :) Yeap, that's it. And if you don't plan to demonstrate how to setup reverse proxy, would be great to hear at least your ideas about handling reverse proxy, during streaming.

Thanks.

[yamalight]

Aight, sounds good 👍

[yatki]

Hey @yamalight Do you plan to continue doing proposals any time in near future ?

[yamalight]

@yatki yes! absolutely! planning to start doing them again once I'm back from vacations in the beginning of Sept :)

[yamalight]

Done in https://github.com/BuildingXwithJS/highly-available-chat

[yamalight]

Video is up on youtube: https://www.youtube.com/watch?v=shJo8Wj7jMA