search

awslabs / gluonts

Probabilistic time series modeling in Python
https://ts.gluon.ai
Apache License 2.0
4.62k stars 753 forks source link

Error in definition of coverage metric? #2736

Closed dwolffram closed 1 year ago

dwolffram commented 1 year ago

Description

I'm wondering why there is a strict inequality in the definition of the (quantile) coverage (https://github.com/awslabs/gluonts/blob/dev/src/gluonts/evaluation/metrics.py#L84):

np.mean(target < forecast)

Shouldn't this be np.mean(target <= forecast)? Right now, if you predict 0 and observe 0, it does not count as covered. Or am I missing something?

Thanks!

lostella commented 1 year ago

Related: the same definition is used in gluonts.ev, see https://github.com/awslabs/gluonts/blob/dev/src/gluonts/ev/stats.py#L50