nkaz001 / hftbacktest

A high-frequency trading and market-making backtesting and trading bot in Python and Rust, which accounts for limit orders, queue positions, and latencies, utilizing full tick data for trades and order books, with real-world crypto market-making examples for Binance Futures
MIT License
2.02k stars 397 forks source link

Kernel crash when using ROIVectorMarketDepthBacktest #157

Closed mmertc closed 1 day ago

mmertc commented 2 weeks ago

Hello,

I'm trying to backtest a strategy using your library but the cell with my hbt.elapse loop always crashes when I use the ROIVectorMarketDepthBacktest. To be specific, If I select roivector it crashes if my loop contains any buy or sell orders; but if I also define a roi_ub and roi_lb it crashes regardless of my loop content, even if my loop is completely empty. The crash contains absolutely no information about its reason. A few of my feature calculations use a significant amount of depth data and I hope that using roivector will contribute to the performance.

Any help is much appreciated.

nkaz001 commented 2 weeks ago

If you can write the code in Rust, it will provide more detailed information when it crashes. Alternatively, if you can share your data and code, I can take a look at it.

mmertc commented 2 weeks ago

from hftbacktest import BacktestAsset, HashMapMarketDepthBacktest, ROIVectorMarketDepthBacktest from hftbacktest import Recorder import numpy as np

asset = ( BacktestAsset() .data(['btcusdt_20241001.npz'])

.data([data_loaded])

    #.initial_snapshot('usdm/btcusdt_20240808_eod.npz')
    .linear_asset(1.0)
    .constant_latency(0, 0)
    .risk_adverse_queue_model()
    .no_partial_fill_exchange()  #TODO: Partially filled orders are not getting canceled!
    .trading_value_fee_model(-0.00005, 0.0007)
    .tick_size(0.01)
    .lot_size(0.00001)
    .last_trades_capacity(20)
    .roi_lb(60000)
    .roi_ub(70000)

)

hbt = HashMapMarketDepthBacktest([asset])

hbt = ROIVectorMarketDepthBacktest([asset])

recorder = Recorder(1, 50_000_000)

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Well, first of all, I dont readily know any method to paste a code snipped beautifully and I will be lazy. The above segment is for generating the asset. The below segment is an essentially empty elapse loop. The kernel consistently crashes when I run the below code. <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

import numpy as np

from numba import njit, uint64, float64 from numba.typed import Dict from hftbacktest import BUY, SELL, GTX, LIMIT

from utils.jit_funcs import float2str

@njit def grid_trade(hbt, recorder):

prev_ask_qty = 0
prev_bid_qty = 0
prev_position = 0

while hbt.elapse(1 * 1e9) == 0:
    a = 5

hbt.close()
return True

grid_trade(hbt, recorder.recorder)

nkaz001 commented 1 week ago

Can you provide the data that you used?

mmertc commented 1 week ago

Yeah sure. I'm just using a tardis sample data for btcusdt spot.

https://datasets.tardis.dev/v1/binance/trades/2024/10/01/BTCUSDT.csv.gz https://datasets.tardis.dev/v1/binance/incremental_book_L2/2024/10/01/BTCUSDT.csv.gz

nkaz001 commented 1 week ago

I will take a look into it.

nkaz001 commented 1 week ago

I found that the price bound was not being checked in the depth clear event. You can see the fix here. I will update the Python repository soon after verifying that the other updates are ready for release. I plan to release the fix by next weekend. Thank you for reporting this issue.

nkaz001 commented 1 day ago

The fix is included in the recently updated version 2.1.1.