A problem with Tardis BYBIT data.

[artiko88]

So i am trying to wok with data from Tardis. So far, all the binance-futures data from Tardis was correct and worked nicely. But now i am working with Bybit and got stuck. First i tought something is wrong with my strategy but more in depth research actually showed that it has something to do with the data itself. Maybe it is converted not correctly, or maybe it contained mistakes from the start at Tardis.

So here is what i get with simple setup:

@njit
def test(hbt):   
    tmp = np.empty((100_000_000, 3), np.float64)
    i = 0
    while hbt.elapse(60_000_000):
        hbt.clear_last_trades()
        tmp[i] = [hbt.current_timestamp, hbt.best_bid, hbt.best_ask]
        i += 1
        if i > 100:
            break
    return tmp[:i]

Output:

Load data/bybit/BCHUSDT/_2023-07-02.npz
bybit-BCHUSDT_Depth-5_Start-20230702-End-20230819
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125
291.125 291.125

When i switch to binance-futures, it spits out actual prices:

Load data/binance-futures/BCHUSDT/_2023-07-02.npz
binance-futures-BCHUSDT_Depth-5_Start-20230702-End-20230819

282.83500000000004 282.83500000000004
282.725 282.725
282.115 282.115
282.165 282.175
281.705 281.705
280.83500000000004 280.845
281.605 281.635
282.235 282.195
281.365 281.21500000000003
281.625 281.575

I tried examining npz file. It does contain information and each row is different, so no repeating values like those above. I got the same output from this simple function both with and without specifying snapshot file.

For the sake of saving time, i also ran checks that we discussed in previous issue, with following results:

Rows where timestamp in column 2 is not bigger than timestamp in column 1:
740
Rows where timestamp in column 1 is not higher than in previous row:
0
Rows where timestamp in column 2 is not higher than in previous row:
0
Rows with zero timestamp:
0

I am confident that i am using correct tick and lot sizes for BCH at BYBIT (0.05, 0.01). Some other coins works nicely. For example, LPTUSD, ADAUSDT, ARBUSDT and others.

Here is a link to converted files: https://drive.google.com/drive/folders/141LdAXZwvY2mXjXuZmQrALL7Zaj1LUCe?usp=sharing

Hopefully, this issue can be resolved. Can't imagine doing the research without this tool.

[artiko88]

Thanks, its working!

[nkaz001]

thank you for the report.