Potential market making rebates accounting issue

[ajecc]

Hi! Thank you for this awesome project. I have a remark regarding the rebates:

After backtesting a strategy using this framework, I decided to test it live using a minimal investment. I collected the live OB and trade data to compare the results of the live trading session with what the backtesting framework would output. After an aprox 12 hour session, I went from 59.13 dollars to 55.89.

I then ran the backteting framework on the same interval with the same data. Everything coincides with the live session (my fair price, half spread, skew, inventory position are all roughly the same). The number of trades and the volume is similar with the live trading session as well. What doesn't coincide is the actual PnL. The backtest framework went from 59.13 to 58.64.

I attached the backtest PnL plot:

The summary:

=========== Summary ===========
Sharpe ratio: -8.6
Sortino ratio: -8.9
Risk return ratio: -244.9
Annualised return: -858.60 %
Max. draw down: 3.51 %
The number of trades per day: 461
Avg. daily trading volume: 0
Avg. daily trading amount: 20021
Max leverage: 8.09
Median leverage: 0.74

I use Bitmex fees, which are -0.015% for linear contracts. All my orders are GTX only. Order size is always the minimum of 0.001 BTC. My HftBacktest call is:

    hbt = HftBacktest(
        [data[-1]],
        tick_size=0.5,
        lot_size=0.001,
        maker_fee=-0.015 / 100,
        taker_fee=0.075 / 100,
        order_latency=ConstantLatency(entry_latency=50000, response_latency=25000),
        queue_model=SquareProbQueueModel(),
        asset_type=Linear,
        trade_list_size=100_00000,
        exchange_model=PartialFillExchange
    )

Ok, so the rebate is 0.015% and the volume is 20021, so the total fees paid back to me are 20021 * 0.015 / 100 = 3.00315. As you can see from the summary, the return excluding fees is aprox -11%, which leaves us with 52.777 at the end of the trading session. Adding up the fees we end up with 55.78, which is very close to my live results, but way less then the backtest results. Using twice the rebate, we end up with aprox 58.7833, which coincides with the backtest result.

Am I misunderstanding something or is the rebate calculation flawed? (If needed, I can send you an email with my code + data.)

[nkaz001]

Thank you for the report. Please send me, I'll look into it.

[nkaz001]

By the way, it's easier to identify the source of differences when you plot your live positions and backtest positions on a single chart, and equity curve as well. Just make sure your resampling window.

[ajecc]

Thanks for the suggestion. It seems like the The number of trades per day: 461 is actually wrong. I counted all the filled orders manually and I ended up with about twice. So the simulation is actually accurate, it's just that the summary is a bit misleading (at least when processing 12 hours of data).

I think the reason why I'm getting bad results live is because my imbalance calculations are a tiny bit outdated, resulting in a bad alpha. Not really sure yet tho. I implemented the live trading algorithm in Python using asyncio, but that might be too slow :/ and I will need to reimplement everything in C++. On a side note: have you been able to deploy production algos using Python or have you resorted to a faster language?

[nkaz001]

Since it shows the daily average trading volume, it seems the number is doubled when you backtest only for 12 hours.

I doubt that slow speed induces discrepancies between backtesting and live trading when you use asyncio and Numba in your calculations; it's reasonably fast. Also, most crypto exchanges have a latency greater than that. In my experience, the major factors affecting the discrepancies are latencies, both feed and order latencies, as well as the queue position model.

I highly recommend plotting both live and backtest positions so that you can identify differences in order execution. Additionally, it's helpful to plot your live alpha and backtest alpha with exact timestamps if you suspect delays in computation.

[ajecc]

@nkaz001 Thank you very much for the suggestions. I will try to capture actualy exchange latencies and use those, as I saw the framework supports that. I will close this issue as I don't think we need it anylonger.