search

VivekPa / AIAlpha

Use unsupervised and supervised learning to predict stocks
MIT License
1.74k stars 448 forks source link

Evaluate the return prediction, not the price prediction #3

Closed sam31415 closed 6 years ago

sam31415 commented 6 years ago

I only went through the readme. The aim stated there is to predict stock returns, so why evaluate the algorithm using stock prices?

Predicting prices with great accuracy is very easy: predict that tomorrow's price will be today's price. The prediction will be correct up to a few percents at most. But that's useless. The analysis in the readme file doesn't make it obvious that the model does better.

I would suggest computing the f1 score or the return-weighted accuracy of a classifier predicting the sign of the returns.

VivekPa commented 6 years ago

added metrics for both returns and prices now.

sam31415 commented 6 years ago

Thanks for the update. Still only reading the readme.md file, it is not very clear that the model learns something. This could be shown by computing a metric for the return predictions and compare it to the metric of the constant model mentioned above. Also, imho it is not very useful to have an actual vs predictions plot, it's hard to see anything there.