Tracking id from order to trade

qacollective commented 3 years ago

Expected Behavior

It is possible to track which order resulted in which trade.

Actual Behavior

Once an order is filled, it is discarded. This is fine, especially to save on resource usage, but there is no way to track the relationships between orders and trades over the long term.

Steps to Reproduce

Run a long backtest which will result in multiple trades being taken out by multiple orders at the same price and tp/sl
Attempt to record exactly what happened during trading for later dissection and detailed reporting

Additional info

Background on this is that all our usual trading software traces an order with an order id and that id persists throughout all the other record types. In backtesting.py's case, this is trades and closed_trades. If similar tracking was possible in backtesting.py, that would enable:

a wealth of existing tools to be used with the data that can be output from this solution
more complex contingent trading algorithms to be tested in backtesting.py
additional data to be stored about each order or trade, depending on trading method

Proposed change

I am willing to submit a pull request containing changes to:

Enable an optional integer parameter for strategy.buy(), strategy.sell() and Trade(), which would be an integer (? or any data type? ) named id
This user supplied number would be accessible as another parameter on both Order and Trade
This user supplied number would be transferred to a Trade when an order is filled and subsequently discarded

However before I do this, I just wanted to check that people (and particularly the author @kernc) think it is a good idea and that there isn't a better way of solving this problem.

Version info

Backtesting version: 0.3.0

Devjeel commented 3 years ago

I was trying something very similar recently, Where I have to change stoploss from active trades for only one of the trades.

Whould be best if buy() / sell() functions takes optional argument of OrderID, that ultimately goes into Trades. With that, later I can modify sl, tp, or cancel the open position(ie. if on long, sell instantly).

qacollective commented 3 years ago

I was trying something very similar recently, Where I have to change stoploss from active trades for only one of the trades.

Yes, it is quite difficult to find an order that resulted in a particular trade. There is no order_price on trade that could be used to figure that out either. So an id would solve multiple problems, including tracking extra detail outside of backtesting.py.

kernc commented 3 years ago

This looks like a duplicate/related to issue https://github.com/kernc/backtesting.py/issues/53.

Can you give an example (code) of some decision/analysis done upon orders/trades with such an id?

I'm definitely more for the arbitrarily-typed label/tag than a strictly numeric id.

Would it be of equal and alternative use to auto-save and propagate the time index when each order was placed?

qacollective commented 3 years ago

This looks like a duplicate/related to issue #53.

Yes, I did look at this issue before raising this one but that appeared to be more about logging the results of backtesting while it was running. Happy if this issue helps though.

Can you give an example (code) of some decision/analysis done upon orders/trades with such an id?

I'd love to, although that's not so easy at the moment. But a couple of more specific examples may help?

A situation where you are instructed to maintain an order or trade at a set price over a set period, only placing the same order again once the trade at that price has closed. At the moment there is not enough information shared between Order and Trade to do this.
A situation where you have multiple mechanisms making trading decisions depending on what phase the market is in. It would be handy to be able to put a reference into each order/trade so you could monitor how each of the trade mechanisms are responding in real time.

I'm definitely more for the arbitrarily-typed label/tag than a strictly numeric id.

That's great because the more I've been thinking about it, me too. If you wanted to hold more information about an order/trade than backtesting.py is designed to, you could link it to that information via an int or forgetting memory concerns, you might even be as cavalier as to put that info into a dict and use that as an id. Leaving that option open certainly maximizes flexibility of use despite what you may think about its potential for misuse.

Would it be of equal and alternative use to auto-save and propagate the time index when each order was placed?

That's actually a great idea. I hadn't yet thought of that. It would be a valuable piece of additional information that could be used as an id which would satisfy the need to have some lineage between Order and Trade. Though your idea does help to crystallize my own: there is also value in the id being external. I can pass some meaning into backtesting.py that has some relevance elsewhere in order to store additional information and potentially make more complicated trading decisions based on that information. Being external also means that I don't have to store another mapping between a backtesing.py id and my id.

I may have been a bit trigger happy with my PRs. The most recent one still has some bugs. I'm currently testing this now by actually using the id parameter. After typing these responses to your comment, I'm thinking I may rename id to external_id.

qacollective commented 3 years ago

FYI to anyone coming across this thread, #200 now seems to be working with new tags functionality.

kernc / backtesting.py