Better server-side management of sockets

[stevenhao]

1) Use socket.io rooms https://socket.io/docs/server-api/#socket-join-room-callback 2) Add monitoring to better understand why connections are getting dropped 3) Scale the backend using something like elastic load balancer and kubernetes

[stevenhao]

This will address https://github.com/downforacross/downforacross.com/issues/110

[stevenhao]

The root cause might be because my server is behind cloudflare.

If you’re intending to use CF websockets, be prepared for random (and potentially massive) connection drops, and be sure you’re architected to handle these disconnects gracefully. Cloudflare rolling restarts have caused hundreds of thousands of websocket connections to be disconnected in a matter of minutes for us. https://news.ycombinator.com/item?id=11635683

So,

4. Don't proxy websocket traffic through cloudflare (use letsencrypt for SSL)

[stevenhao]

More updates on research into cloudflare websocket support: https://techxperiment.blogspot.com/2020/06/aws-ec2-tomcat-jsr-356-secure.html https://community.cloudflare.com/t/websockets-disconnected-in-aws-tokyo/44680 https://stackoverflow.com/questions/39668410/whats-disconnecting-my-websocket-connection-cloudflare-apaches-mod-proxy

tl;dr: cloudflare websocket support is not very good.

moving off cloudflare is probably going to resolve multiple issues ("blank loading page" and "losing sync with server").

[stevenhao]

This is done