Proper backtesting (stop/limit orders, trades based on bid/ask, etc)

[lefnire]

Current implementation is very crude - just enough to ballpark strategy & hyper combo. Backtesting is custom-built. It relies on next-state close prices for PNL; buys/sells on close (instead of buying at ASK and selling at BID); only supports market-orders as such (no stop/limits); etc. Then there's more complex bits like order depth (how much can I buy at this price before that guy runs out and I have to fill the rest of that order at a different price). Very importantly, w/o stop/limit orders, there's no risk-control, which is half the game (could double the agent's performance). (Accounting for order types will require augmenting the Env's action-space).

All this could be solved w/ a backtesting library, and it's high-time. I've scoped out a few, my notes here. I'm leaning on backtrader currently, and possibly paired with ccxt

[s2c]

Have you looked at btgym @ https://github.com/Kismuz/btgym

I'm not 100% sure how it works yet, but it might be useful. The repo author has some implemented examples.

[methenol]

backtrader seems like a solid framework. Toyed with btgym that runs off of backtrader a bit but ran into issues using external data and didn't sink the time into changing the formatting to get it to play nice, but the goal was to pull from ccxt. The unreal LSTM strat in that repository is interesting.

Something to consider with the order types, not all exchanges support executing 'advanced' orders like that through their APIs (I believe kraken supports stop loss order, poloniex does not), so changing it at the level of what's sent to the API runs risk of limiting the supported exchanges. It seems like something that can easily be done via code, but should probably go into a separate process responsible for executing buy/sell orders and monitoring the ticker/order book.

Depending on the time scale you're looking at, time lag can turn a good buy/sell decision into a very bad one, pretty quickly. In an exchange with a maker/taker fee structure (most), we want maker as close to every time as we can to save those percents. That being said, there are times when taking a hit on the taker fee to secure an asset when the anticipated returns make up for the loss is better than not securing it at all.

Instead of doing highest bid or lowest ask, I take the highest bid and the bid below it, take the difference of those two and add it to the highest bid (same thing for the lowest ask), this pretty much guarantees slipping it in on a maker fee and puts it at the top of the book giving it a good chance of getting satisfied, then it's just a matter of refreshing the order if it doesn't get satisfied and the price shifts (have to determine if it's still a good decision at that point). When this gets screwy is when the spread between bids and asks is temporarily high due to volatility, might be worthwhile to wait a few ticks or base the order off the average spread prior to the execution tick?

Has anyone stumbled on any projects that feed the order book into a deep learning framework? I don't think the data is available historically through APIs which would limit the data to feed into backtesting, so it couldn't really be used as an 'indicator' in that sense, but feel it could be useful at least in the execution phase