awslabs / gluonts

Probabilistic time series modeling in Python
https://ts.gluon.ai
Apache License 2.0
4.54k stars 747 forks source link

Inconsistent error definition #2272

Open codingWhale13 opened 2 years ago

codingWhale13 commented 2 years ago

Currently, the meaning of error doesn't seem consistent. Concretely, MSE uses target - forecast.mean as error while other metrics (absolute error, MAPE, ...) define the error as target - forecast.median.

Is this a conscious decision or would it make sense to stick to one definition?

For context, see here.

lostella commented 2 years ago

It's a conscious decision. Under the assumption that the predicted distribution (forecast) is what target is sampled from, then:

jaheba commented 2 years ago

Maybe something we can do is to document this better. So for each metric using mean or median we could highlight which one is used and why.