nkaz001 / hftbacktest

A high-frequency trading and market-making backtesting and trading bot in Python and Rust, which accounts for limit orders, queue positions, and latencies, utilizing full tick data for trades and order books, with real-world crypto market-making examples for Binance Futures
MIT License
2.01k stars 395 forks source link

Different content for depthUpdate data #155

Open qihaiqianqiu opened 3 weeks ago

qihaiqianqiu commented 3 weeks ago

Hi I see from the API and sample data that depthUpdate data in binance should looks like: 60228023037049 {"stream":"btcusdt@depth@0ms","data":{"e":"depthUpdate","E":1660228023941,"T":1660228023931,"s":"BTCUSDT","U":1801732831593,"u":1801732832589,"pu":1801732831561,"b":[["2467.10","0.000"],....,"a":[["24653.60","0.000"],...}}

But neither do I download from collector nor from binance socket api doc says the same. We have data without "T"(timestamp) and also tagged as @depth@100ms

{
  "e": "depthUpdate", // Event type
  "E": 1672515782136,     // Event time
  "s": "BNBBTC",      // Symbol
  "U": 157,           // First update ID in event
  "u": 160,           // Final update ID in event
  "b": [              // Bids to be updated
    [
      "0.0024",       // Price level to be updated
      "10"            // Quantity
    ]
  ],
  "a": [              // Asks to be updated
    [
      "0.0026",       // Price level to be updated
      "100"           // Quantity
    ]
  ]
}

As a result, convert is not applicable and the timestamp is missing. Is there anything wrong on my set up? I just directly build collector without any modification and dont know how to get 0ms depthUpdate

nkaz001 commented 3 weeks ago

The example provided is based on Binance Futures, whereas your reference pertains to Binance Spot. Binance Spot does not have a transaction time (the exact time when events like matching occur at the exchange), so estimation is required. A simple approach is to subtract a certain amount of latency from the event timestamp (the time when the exchange sends the data). Additionally, the market depth frequency differs; you need to combine the market depth data with L1 (bookTicker) to obtain more frequent updates.

qihaiqianqiu commented 1 week ago

I think I understand your answer. So for spot, @trade and @bookticker are real-time but without exchange timestamp, @depth is also without exchange timestamp, meanwhile is updated in 1000ms/100ms. As a result, to retrive the tick-by-tick data, we need to add latency and combine them together. For futures, all is real-time with a exchange timestamp, so the only thing we need to do is sort them by time and maintain a orderbook with a inital snapshot. Am I get it right? thank you so much for your comprehensive answer!

nkaz001 commented 6 days ago

Yes, you're correct. As far as I remember, actually Binance Futures also needs to combine bookTicker and depth streams to get the most frequent updates for the BBO. You can check it yourself by comparing the BBO with the timestamp from the bookTicker stream and the depth stream.