search

tardis-dev / tardis-machine

Locally runnable server with built-in data caching, providing both tick-level historical and consolidated real-time cryptocurrency market data via HTTP and WebSocket APIs
https://docs.tardis.dev/api/tardis-machine
Mozilla Public License 2.0
243 stars 48 forks source link

WS replay - "replaySessionKey" may not be removed, blocking new connections #1

Closed alon-e closed 4 years ago

alon-e commented 4 years ago

WS session keys are used to aggregate connections (key={$from}-{$to}), whoever, if a connection is not closed nicely, it's impossible to reestablish a connection with the same from and to. also, the same goes for 2 different exchanges, not in the 5sec time-window before the session starts, this mechanism doesn't feel robust enough.

to reproduce:

  1. establish 1 connection to FTX for "2020-01-01T00:00:00.000Z", "2020-01-02T00:00:00.000Z"
  2. kill the script above and restart it. you will get (and a close on the WS): 
    2020-02-18T14:17:15.216Z tardis-dev:machine trying to add new WS connection to replay session that already started

    the same with another exchange, even if still first exchange still running properly.

the only way I could reconnect with the same from-to was by consuming all the data till the server hits on_finished

https://github.com/tardis-dev/tardis-machine/blob/4031ada47aeb35cde550dae1fe41186555726fc2/src/ws/replay.ts#L29-L32

thaaddeus commented 4 years ago

@alon-e apologies for the late reply, I must have missed notification. It has been fixed over a week ago in https://github.com/tardis-dev/tardis-machine/commit/5d27737f9014ea81a414ca0c61bb028ff1b9b1db and released in v2.0.5 Mechanism has been changed to explicitly requesting 'synchronized' session for multiple connections by sending session=you-session-key query string param.