search

altangent / ccxws

WebSocket client for 38 cryptocurrency exchanges
MIT License
617 stars 186 forks source link

Is it possible to listen for heartbeat messages with this library? #185

Closed Slurpgoose closed 4 years ago

Slurpgoose commented 4 years ago

Hi,

Thank you for maintaining this project it has made working with multiple exchange web socket connections a lot simpler. I just have a couple questions:

  1. is it possible to receive heartbeat messages for the exchanges that support it using this library?

The CCXWS socket client performs automatic reconnection when there are disconnections. It also has silent reconnection logic to assist when no data has been seen by the client but the socket remains open.

  1. is there a recommended method to track if trades are missed during silent disconnects, reconnects?
  1. is there a recommended strategy for re-syncing state on script failure?

Prior to using this library I would check to make sure I would receive at least 1 heartbeat/ s per market to quickly detect if something is wrong.

For coinbase pro my re-synchronization strategy has been to backup state to redis a couple times a second and on script failure to retrieve the last id and current candle from redis store, then when the market receives a new trade event to validate the difference between old tradeId and new tradeId is not greater than 1. then using the exchange API to fill in the missing gaps. while this strategy works for Coinbase Pro, it does not for exchanges that do not use sequential ids.

bmancini55 commented 4 years ago

Hi thanks for submitting the issue.

Heartbeats are currently not being used by the library but it is something that would be a good addition in an upcoming refactor #149. Heartbeats when supported could be an input the reconnection watcher to provide more accurate detection of dropped connections.

Right now, the watcher will trigger a restart if there is no message activity for 90 seconds. 90s is very high for an active exchange with many active subscriptions, but might cause repeated reconnections if you are only subscribed to a trade feed for an illiquid market. Unfortunately this isn't very tune-able at the instantiation level either, which is an easily resolvable gap in the library.

IMO, the best way to detect drops is to subscribe to the disconnected and connected events. When a disconnected event is triggered you should perform a REST request to retrieve any missing trades. I would continue this polling until you receive a connected event.

We also have a periodic backfill process that uses REST to reconcile anything that might have been missed by socket feeds.

Slurpgoose commented 4 years ago

that makes a lot of sense, im currently subscribing to multiple illiquid markets on bittrex so its likely an issue I will be running into in the future.

At the same time the current watcher implementation has been running stable for me and local candle store seems to match exchange historical response at least for a few active markets ive manually checked.

I have not been utilizing the disconnected event, so I will be sure to implement some logic for it now. unfortunately I will still need to find a better solution to track missed trades by market to avoid running into request limits/long resync periods.

I have migrated my project to using this library from single exchange libraries for (coinbase, bittrex) and currently reading through kraken, binance docs. I would love to contribute in the future when I get more familiar with this project.

thanks again for the response, cheers

bmancini55 commented 4 years ago

This latest change which will be release shortly includes the watcherMs option when you instantiate a client. This should allow you to customize the reconnection.

The use of heartbeat is being tracked in #149, so I'm going to go ahead and close this issue. Thanks!