nkaz001 / hftbacktest

A high-frequency trading and market-making backtesting tool in Python and Rust, which accounts for limit orders, queue positions, and latencies, utilizing full tick data for trades and order books, with real-world crypto market-making examples for Binance Futures
MIT License
1.78k stars 357 forks source link

Supporting other exchanges #94

Open garyttierney opened 2 months ago

garyttierney commented 2 months ago

Hi! Amazing work putting this together. I've been following the Rust port from a distance for a month or so and I'm finally getting my feet wet with hftbacktest :).

I'm adding support for Kraken spot markets right now and in the process of that I realised there's currently no good specification of what a Connector should do (i.e. the LiveEvents it should emit, and when). It's been simple enough to piece together by looking at existing Connectors but I wondered if you think with a 3rd exchange added it might be time to start unifying connector logic. E.g., consolidating existing connectors into something like (rough API, I'm sure it needs a bunch of changes):

pub trait Broker {
    async fn submit_orders(...);

    async fn cancel_orders(...);

    async fn get_positions(...);

    async fn subscribe_to_asset(...) -> impl Stream<Item=L2OrderBookEvent>;
}

It would allow connector implementations to share nice enhancements like rate-limiting approaches and would make the overhead of adding a new exchange/broker a little easier by centralizing some of the bookkeeping logic that is done by BinanceFutures, Bybit, and eventually my Kraken connector.

Alternatively, I'll follow the same approach as was done for Bybit (and maybe produce some accompanying documentation on adding support for a new exchange in the process).

nkaz001 commented 2 months ago

Thank you for your interest in contributing to this project.

The current structure to support a live connector is somewhat awkward. This is mainly because the project was originally built for backtesting and later adapted to support live trading with the same codebase. Since it is optimized for backtesting, it cannot use coroutines.

Consequently, the current Connector trait doesn't have async functions. Position updates need to be handled through the WebSocket stream and delivered through the channel so the bot recognizes the position as a state without a function call. Additionally, subscriptions should be set up before running the bot.

I am not confident in the current structure. But, the main goal is to maintain backtesting performance while ensuring compatibility for live trading using the same code.

I think we may need to add another intermediary interface. This way, if a bot submits an order (sync), it's transferred through the channel, and another module manages the connector through the trait (async).

Please contact me via Discord so we can discuss the best structure.

bohblue2 commented 2 months ago

I am not confident in the current structure. But, the main goal is to maintain backtesting performance while ensuring compatibility for live trading using the same code.

I think we may need to add another intermediary interface. This way, if a bot submits an order (sync), it's transferred through the channel, and another module manages the connector through the trait (async).

Please contact me via Discord so we can discuss the best structure.

I agree 100% with this comment. We should consider restructuring the Market Connectivity part to allow for more sophisticated order state management.