search

geneura-papers / 2015_books

Paper on the book publishing prediction
Artistic License 2.0
2 stars 0 forks source link

Forecasting Errors Method #26

Closed deantares closed 8 years ago

deantares commented 8 years ago

Hi everyone,

I have been reading the paper and I have a question about the methods used for the measure of the forecasting error. We are only forecasting a single value, not a time serie. So i think that we can not use RAE and RRSE. Both methods, depend of the previous value of the time serie (pi-1).

No?

7ossam81 commented 8 years ago

Thanks @deantares for the question. Actually, both measurements don't depend on the previous value of the prediction. Using (p_{i-1}) is wrong. This should be the average of the actual values instead. I will fix that :)

pacastillo commented 8 years ago

Thanks!! I made a mistake when I typed the text...

deantares commented 8 years ago

You are welcome. But I think that if you use the average then you are using MAPE (Mean Absolute Percent Error) not RAE. No?

All the stuff in error metrics are horrible...

7ossam81 commented 8 years ago

No. MAPE is a different error measurement. In MAPE you divide by only the actual value while in RAE you divide by (average - actual).

Have a look at this [wikipedia:MAPE] and this [How to interpret error measures in Weka output?]

For RAE and RRSE, in general, the smaller values are better, while values less than 100% mean that you are performing better than just predicting the average. So in our case for M5P for example something like 40% ~ 50% I expect to be useful :)

vrivas commented 8 years ago

Hi! Maybe you are interested in Rob J. Hyndman and Anne B. Koehler, "Another look at measures of forecast accuracy", International Journal of Forecasting (2006). 22(4), 679-688. http://robjhyndman.com/papers/another-look-at-measures-of-forecast-accuracy/

I wrote a library in JavaScript (of course) to compute lot of error measures according to that paper: https://github.com/vrivas/ts-error-measures

Cheers!

2016-05-05 0:28 GMT+02:00 7ossam81 notifications@github.com:

No. MAPE is a different error measurement. In MAPE you divide by only the actual value while in RAE you divide by (average - actual).

Have a look at this [wikipedia:MAPE https://en.wikipedia.org/wiki/Mean_absolute_percentage_error] and this [How to interpret error measures in Weka output? http://stats.stackexchange.com/questions/131267/how-to-interpret-error-measures-in-weka-output ]

For RAE and RRSE, in general, the smaller values are better, while values less than 100% mean that you are performing better than just predicting the average. So in our case for M5P for example something like 40% ~ 50% I expect to be useful :)

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/geneura-papers/2015_books/issues/26#issuecomment-217022944

Victor Manuel Rivas Santos

JJ commented 8 years ago

Víctor, you can use Figshare to get a DOI for a library, so that it can be referenced. Do it, by all means.