Order Book Sequence Number Validation

[bmoscon]

Some exchanges support methods for ensuring no messages were lost, some do not. This is especially important for exchanges that supply only orderbook updates. All exchanges need to be re-checked to see if this support exists or not, and to validate that the existing code validates correctly (assuming said code exists):

• [x] Bitcoin.com
• [x] Bitfinex
• [x] BitMax - has sequence numbers in API v2, but they are not sequential rendering them somewhat useless
• [x] Bitstamp - L3 book is full book every update (so no possibility of an incorrect book from a missed delta), but L3 is only top 100 orders from each side. L2 has complete (unlcear from docs if its the complete book, but its more than 100 entries per side). No sequence numbers or other way to ID a dropped message. Bitstamp does offer an L2 book that is top 10o per side each update (no deltas). Unclear if this would be better to support or not.
• [x] Bittrex - new v3 api supports sequence numbers
• [x] Blockchain.com - supports sequence numbers, cryptofeed correctly validates.
• [x] Bybit - no sequence numbers - verified 09/01/2020
• [x] Binance
• [x] Binance Jersey
• [x] Binance US
• [x] BitMEX - No sequence numbers - verified in docs on 8/24/2020
• [x] Coinbase
• [x] Deribit
• [x] EXX - no sequence numbers - verified in docs 09/01/2020
• [x] FTX - FTX and FTX US provide a checksum. Checksum validation optionally enabled (disabled by default)
• [x] FTX US
• [x] Gate.io - no sequence numbers - confirmed in docs on 8/24/2020
• [x] Gemini - no sequence numbers on v2 of the api, on v1, sequence numbers are supported, so would need to move back to the older api, which is harder to support, to enable sequence number verification
• [x] HitBTC - exchange provides sequence numbers, code updated to utilize them
• [x] Huobi - huobi currently is using complete snapshots, not incremental updates. SeqNumbers are only available for incremental updates
• [x] Huobi DM
• [x] Huobi Swap
• [x] Kraken - checksum validation supported in cryptofeed. Is disabled by default, but can be enabled
• [x] Kraken Futures - sequence number checks added
• [x] OKCoin - supports checksum validation
• [x] OKEx - supports checksum validation
• [x] Poloniex - sequence numbers supplied - reconfirmed that the code matches the docs
• [x] Upbit - I don't speak/read korean, so I'm relying on google translate on their docs - I can't find any orderbook references in the docs, but looking at the raw data from the exchange, its sending complete books each time, not deltas. No sequence numbers appear in the message (as best I can tell).

[bmoscon]

Adding checksum verification to FTX takes about 0.4 ms (400 microseconds).

[zahnz]

Bybit has a "cross_seq" field which looks like a sequence number, but based on messages in their API support Telegram channel, it's clear that it's not a sequence number, and is only used internally: That screenshot is from 2020-Feb-21, but the same kind of response is consistently given to other users in the support channel who've asked the similar questions over the last 6 months. There's no indication in the API change announcement Telegram channel that this has changed either.

[bmoscon]

confirmed field is not useable in bybit - https://github.com/bybit-exchange/docs/issues/102

[bmoscon]

adding checksums to kraken adds ~750 microseconds (0.75 ms)

[bmoscon]

checksum validation for okex/okcoin takes 0.4ms

[bmoscon]

all exchanges verified, closing

[DennyZen]

i am using Bitmex. if they dont have sequence numbers - can i get new orderbook snapshot by rest periodically and check it ? @bmoscon

[bmoscon]

@DennyZen you can, but it would be hard to know how far off the rest book was from the websocket book, its unlikely they'd be identical at any given moment, you'd have to look backwards at an older websocket book