SocketCluster / socketcluster

Highly scalable realtime pub/sub and RPC framework
https://socketcluster.io
MIT License
6.15k stars 314 forks source link

Callbacks scale much more than async/await approach #590

Open Mukund2900 opened 1 year ago

Mukund2900 commented 1 year ago

I had done some extensive testing, I had some observations ->

The event loop is not able to handle when we have around 30k average connections with the above mentioned condition, whereas same works amazingly well with 14.2 version.

Mukund2900 commented 1 year ago

Async await works very well when we work on a dev environment, due to ease of use and understanding. Even when load is not much it looks very promising, until there is an actual load. Obviously if we increase the number of machine it will work fine. But when benchmarking with same setup callbacks scale much better.

jondubois commented 1 year ago

When running in v14, do you run as multiple worker processes? In v14, it would scale across multiple processes on the same host so it can be a factor which could make it look like it performs better (though in reality it just has more capacity). This approach was abandoned as it has some disadvantages when used with tools like Docker and Kubernetes which tend to encourage single-process instances on the back end (it simplifies monitoring). It's possible that async/await can use more CPU though it should be possible to minimize the impact by optimizing how it's used in your application logic (e.g. avoid making too many expensive I/O calls like database lookups during connection handshake).

Note that you can run multiple scc-worker instances on a single host on different ports; if the host has enough CPU cores, they won't slow each other down. You can also launch multiple scc-broker instances as required and you also need an scc-state instance to coordinate. So it would act like a cluster but on a single host. You can also use an additional load balancer like nginx, apache or haproxy listening on port 443 (or 80) to forward WebSocket connections to your SC instances which are running on other ports on the same host. A good approach is to make the load balancer act as a terminating proxy for SSL/TLS so that it handles encryption and forwards plain traffic to the SC worker instances (that way you don't need the SC instances to be configured for HTTPS/WSS).

You can read the SCC guide here: https://github.com/SocketCluster/socketcluster/blob/master/scc-guide.md - but instead of setting up your cluster on multiple hosts as described, you can set it up on a single host; it works essentially the same way. It scales well (linearly) on a single large machine if you have no more than 1 process per CPU core.