Result or Statistic class

ryankennedyio commented 8 years ago

As mentioned in: https://www.quantstart.com/articles/Advanced-Trading-Infrastructure-Portfolio-Class

What's missing from this list so far? Perhaps the most important missing piece is any mechanism for calculating trade strategy statistics and viewing the results. This includes performance metrics like Sharpe Ratio and Maximum Drawdown, as well as an equity curve, returns profile and drawdown curve.

Rather than strongly-coupling the results to the PortfolioHandler class, as in the previous QSForex and the event-driven backtester codes, we are going to generate a Results or Statistics class that will calculate and store the necessary performance metrics based on the results of a backtest. We can then use these classes to produce further "client" utilities, such as a web interface or GUI tool, to view the results of a backtest.

Has any work or planning been done on this class? I'll get started if not.

There is currently an equity_file written as part of the Backtest._run_backtest method, but it only tracks portfolio equity over time. Not sure if this is the "final" solution? Should this be tracking more information, and then the Statistics class creates visualisations based on the data in that file? Or should the Statistics class keep more information in-memory?

I'm thinking that a Statistics class might "track" the portfolio (cash, equity value, PnL) as events come through the system. Having these Statistics accessible by other classes would also allow them to be incorporated into strategies, which I think would be quite useful.

To implement this, I think that inside the event loop, we do Statistics.update(). This method will update the equity curve, PnL, cash levels, drawdowns, etc etc etc. I don't like the idea of doing this for every single event that comes through though... ?

Only issue I can see so far is that the unrealised & realised PnL is only calculated when a Position is modified. Might make the equity curves look a bit jumpy for less active strategies ?

Also unsure if Logging should be part of the Statistics class -- I think they would share a lot of functionality.

mhallsmoore commented 8 years ago

Ryan,

Thank you for suggesting so many good ideas!

I'll try and answer the questions in order:

I've not carried out any real work on the Statistics class yet. The only available code is that from QSForex (which we should borrow from, if there's any useful aspects, if only to save time).
You're right, it's far from the final solution. This was designed as a simple mechanism to get people started. The file aspect works great for backtests as we're generally only interested in the results right at the end, so having in memory results for backtests is not necessary. However, for live trading it would certainly be great to have an in-memory store of the calculations. We'll need to come up with on-line mechanisms for calculating each of the values, i.e. on-line Sharpe, on-line max drawdown etc. If they're stored in memory then it is relatively straightforward for some client interface to come and obtain them somehow.
I do like the idea of a Statistics.update() method, but equally I agree that it may be a bit excessive to update them on every event, in an HFT situation. This really comes down to what sort of frequency users will be trading at. I was intending, in live trading, to have the system ping the broker for market data between 1-10 minutes, which is the rough "heartbeat" of the system, but of course this is for my personal usage, others may want much less or much more. If we start considering second and sub-second trading, then we'd be doing a lot of extra work.
I agree, in that case we should update the PnL calculations upon new market data input. I'll make a separate issue for this.
I have actually done quite a bit with logging in QSForex, but as yet have not included it in QSTrader. If you have any suggestions about what you would like to see in this regard, let me know.

If you're happy to have a first go at the Statistics module, then feel free to submit a pull request. I'll be adding in other requests this week and working on the software myself, although I'll steer clear of the Statistics class for now.

Thanks again for the suggestions.

Mike.

Anjum48 commented 8 years ago

ffn (http://pmorissette.github.io/ffn/) is a great tool for generating statistics - I use it a lot in my backtesting platform. You could add this as an optional dependency.

mhallsmoore commented 8 years ago

Anjum,

That looks like an extremely useful library. I'll try and make use of it, if possible.

Mike.

mhallsmoore commented 8 years ago

Ryan,

I've now added your Statistics class to the master branch. It seems to work fine on my system.

Thanks,

Mike.

vishnuvr commented 8 years ago

https://github.com/QuriQuant/qq-pat . Looks interesting.

ryankennedyio commented 8 years ago

Cool, happy to consider the issue closed as it's now semi-functional at least.

ffn looks pretty cool actually.

@mhallsmoore would it be possible to set up a longer-term branch to work on the other features outlined for this class ? I don't think with Github it's possible for me to fork, branch and then push the branch back to the original repo -- I think that the branch needs to be done from the original.

mhallsmoore / qstrader

Result or Statistic class #6