Add real-time update system for continuous liquidity data

[dib542]

This could be through websockets, or long-polling, or HTTP2 Server Sent Events (SSE).

In my opinion web sockets aren't as scalable as the other options which all use standard HTTP requests. There used to be an issue with limited connections of websockets over HTTP1 (due to no multiplexing for websockets) but that appears to be resolved with HTTP2. The main challenge around websockets is in a production environment with many indexer instances: keeping connections open, restarting connections, what to do when errors occurs, and what do you do when you roll out a code/architecture update and everybody reconnects at once.

I think we will start with the simplest options first:

• currently (short polling, sometimes)
• next long polling
• future: websockets or HTTP2 SSE
  • I think SSE may be ideal here as we don't need bi-directional communication, the client requests a resource and multiple update responses can be received, potentially replacing all useInifiniteQuery usage on the frontend where we iterate through all available pages of data for several resources.

notes:

• Websockets:
  • https://tsh.io/blog/how-to-scale-websocket/
  • https://ably.com/blog/websockets-horizontal-vs-vertical-scaling
• HTTP2: listening for SSE updates:
  • https://developer.mozilla.org/en-US/docs/Web/API/EventSource#examples