search

BTCMarkets / API

API
120 stars 30 forks source link

Websocket V2 Update cadence is highly limited #160

Closed liamwhittle closed 4 years ago

liamwhittle commented 5 years ago

If I subscribe to the channels ["orderbook"] and subscribe to every market-id:

['BTC-AUD', 'LTC-AUD', 'ETH-AUD', 'ETC-AUD', 'XRP-AUD', 'BCHABC-AUD', 'BCHSV-AUD', 'POWR-AUD', 'GNT-AUD', 'XLM-AUD', 'BAT-AUD', 'OMG-AUD', 'LTC-BTC', 'ETH-BTC', 'ETC-BTC', 'XRP-BTC', 'BCHABC-BTC', 'BCHSV-BTC', 'POWR-BTC', 'GNT-BTC', 'XLM-BTC', 'BAT-BTC', 'OMG-BTC']

Then the number of updates I receive are significantly less than the updates that are actually occurring. For instances I could test this by creating an order which beats the best-bid in a market, for instance, 'POWR-BTC', and the web-socket would not return any update for that market, possibly for many seconds. This behavior is consistent with the sample code I have tried for both Python and Node.js.

Experimentally, the maximum update frequency appears to be about 190 BPM. This would be fine if each message contained multiple markets/ channels of information but it only gives one at a time so many markets are left un-updated.

Thank you for your time

martin-nginio commented 5 years ago

Hi @liamwhittle

Thanks for your feedback.

As far as I'm aware we do not have any restrictions on number of messages (or total data) that is published to specific clients via WebSocket so the case you are describing needs some further assessment from our team to see if we can find any issues.

It's possible that client side bandwidth also play a role here meaning that if your internet connection can only consume up to certain amount of data in a time window then you may not get all the messages. But I can't confirm this until our team performs further validation. If you have the setup it's worth testing it with higher bandwidth as well to see if it makes any difference.

Please also note that our team has planned to develop a new orderbook channel that is published via WebSocket that is much lighter and consumes a lot less bandwidth. This new channel will only publishes orderbook changes instead of the full orderbook data every time that is useful in your scenario. The integration with this new channel requires a bit more logic from your end to replicate the full orderbook at your end (and re-sync it with an API call when needed) but it's more efficient way of publishing events in large numbers. At this stage I don't have ETA for this improvement but it's part of our roadmap for WebSocket v2.

Thanks.

Regards, Martin

liamwhittle commented 5 years ago

Thanks for your timely response Martin, I really appreciate it. I have been using the sample python code exactly as it is given, on a server with plenty of bandwidth. I will write some scripts to test my hypothesis and let you know of my results. Kind regards, Liam

nickchr commented 4 years ago

Hi @liamwhittle

Thanks for your feedback.

... Please also note that our team has planned to develop a new orderbook channel that is published via WebSocket that is much lighter and consumes a lot less bandwidth. This new channel will only publishes orderbook changes instead of the full orderbook data every time that is useful in your scenario. The integration with this new channel requires a bit more logic from your end to replicate the full orderbook at your end (and re-sync it with an API call when needed) but it's more efficient way of publishing events in large numbers. At this stage I don't have ETA for this improvement but it's part of our roadmap for WebSocket v2.

Thanks.

Regards, Martin

Has this Channel been implemented in v2, I don't see the functionality documented in the API? I would like to maintain a real-time L2 orderbook through the WS but I don't see how this is currently possible...

Nick.

martin-nginio commented 4 years ago

Hi @liamwhittle

Thanks for your patience.

While we had this feature in our roadmap but the development was not prioritized.

Let me push for a timeline for delivery. I'm hoping we can deliver this feature during November, but let me confirm the timeframe later this week.

Thanks.

Regards, Martin

martin-nginio commented 4 years ago

Confirming we have added this to our dev roadmap aiming to deliver by 1st of December.

Thanks.

martin-nginio commented 4 years ago

Hi @liamwhittle

This is to confirm that a new WebSocket message type was added today that broadcasts orderbook updates instead of the entire orderbook.

The documentation will be updated shortly but in the meantime, if you want to take a look below covers a basic description of this new WebSocket message type:

The new message type, orderbookUpdate, broadcasts a full snapshot of the orderbook at first (immediately after subscription), and then the subsequent messages only provide updates to the orderbook.

Message format for the initial snapshot:

{"marketId":"LTC-AUD","snapshot":true,"timestamp":"2020-01-08T19:47:13.986Z","snapshotId":1578512833978000,"bids":[["9.57","0.0525",1],["7.62","200",2],["0.07","0.999",1],["6.7","0.99",1],["64.8","7",1], "asks":[["100","93.79257849",6],["1000","86.32619878",22]]}
(reduced number of bids/asks for simplicity)

Message format for subsequent orderbook updates: The message format for orderbookUpdate is the same as the initial snapshot except that it does not have the element "snapshot":true in it. 

{"marketId":"LTC-AUD","timestamp":"2020-01-08T19:47:24.054Z","snapshotId":1578512844045000,"bids":[],"asks":[["65.2","0",0]],"messageType":"orderbookUpdate"}

Notes:

If you have any questions, please let us know.

Regards, Martin

martin-nginio commented 4 years ago

Hi @liamwhittle and @nickchr

I'm closing this ticket as the new orderbookUpdate has been in production for some time now and being used by some of our customers.

Please find below the documentation: https://api.btcmarkets.net/doc/v3/#section/OrderbookUpdate-event

If you have any questions or issues please feel free to open a new ticket.

Thanks.

Regards, Martin