How to scale across multiple servers?

[simpleusr]

Hi,

I am planning to use this awesome project in our standalone application.

The Problem:

The application is scaled out on multiple servers. How should I handle websockets in this picture (The sender may not be connected to the receiver) ?.

As far as I understand, common practice is to use a publish/subscribe broker like RabbitMQ, ActiveMQ, Redis and etc.

What is the recommended practice in terms of this project? I searched for samples but could not find any.

Any comment that would put me on the correct direction will also be appreciated.

Thnx

[PhilipRoman]

Scaling with websockets shouldn't be any different than with other protocols. Just put a load balancer in front of them :P I'm not sure what you mean by "The sender may not be connected to the receiver". Websockets are typically used for real-time traffic such as live updating chat, browser games, etc.

[simpleusr]

Hi @PhilipRoman

Thnx for your response. As far as I understand, connection structure of websockets differ from HTTP. Since http is request/response based, usage of load balancers makes sense because the connection exists only until the response has been given. There is no need to reuse the exact same connection for two different http requests (although keep-alive may keep the same connection)

On the other hand, as far as I understand, for websocket communication the client opens up a connection to the server and reuses it. This connection is long lived and both the server and client use this single connection for communication. For that reason, it seems that there is state in websocket interaction. There should be some statefull data on the websocket server for each open client connection. When servers are horizontally scaled, there should be some mechanism to manage/distribute this state to handle server failover, scaling and etc.

For your load balancer suggestion, are you saying that we should configure load balancers with something similar to sticky sessions?

Maybe the following posts better explain my problem:

https://hackernoon.com/scaling-websockets-9a31497af051

https://medium.com/javarevisited/scaling-websockets-in-spring-services-27023f59868c

Thnx

[PhilipRoman]

Ah, now I see what you meant. Your solution will probably depend on how many users you expect to be in a single "group" that you want to broadcast updates to. The important question is - do you expect to have more users in such a "group" than a single server can handle? If no, then you can ignore all the pub/sub stuff and simply handle everything in-process (saving state to a database to be able to restart and migrate instances).

This was the solution I used. In my case, I was hosting a multiplayer game (with a soft upper limit of 100 users per lobby). Each server reports its current capacity to the load balancer. When a new user needs to be added (or a new lobby needs to be created) load balancer picks nearest server that has enough capacity. If a server crashes, becomes too laggy or some lobby exceeds the server's capacity, the saved lobby state is offered to the load balancer, which can re-assign it to another server and migrate existing connections seamlessly (although the clients can also auto-reconnect to a lobby token if needed so seamless connection migration is optional). But there is no communication between servers (aside from load balancing info).

If you are making something that has less latency requirements and more users per "lobby" than a single server can handle, pub/sub sounds like a good idea.

Either way, this library won't provide any features specific to load balancing, so it's up to you to implement it.

[simpleusr]

Hi @PhilipRoman

Thx for the clarification.