TheRacetrack / racetrack

An opinionated framework for deploying, managing, and serving application workloads
https://theracetrack.github.io/racetrack/
Apache License 2.0
28 stars 5 forks source link

Make Dashboard's real-time stream work with multiple replicas #399

Closed iszulcdeepsense closed 5 months ago

iszulcdeepsense commented 6 months ago

Currently, dashboard's auto-update feature (aka "auto-DDoS") can't connect to Lifecycle's events stream when Lifecycle runs on more than one replica. Usually it ends up with an infinite reconnection loop. Client connects to Lifecycle's socket.io server and gets the session ID, then with the next request (btw: why there are many requests being made?), the other replica handles it and doesn't know that session, thus rejecting it. This reconnection loop was causing a lot of fuss and had to be disabled on Dashboard.

Dashboard and Lifecycle didn't use raw websockets, but socket.io library. Now it's been switched to use use primal websocket protocol instead, which makes browser to keep just one connection to the Lifecycle server.

Dashboard's auto-update feature works with multiple Lifecycle replicas. Live updates on the job list are enabled.