Performance Metric for Chronos

amazon-science / chronos-forecasting

Chronos: Pretrained (Language) Models for Probabilistic Time Series Forecasting

https://arxiv.org/abs/2403.07815

Apache License 2.0

2.02k stars 238 forks source link

Performance Metric for Chronos #96

Closed yeongnamtan closed 3 weeks ago

yeongnamtan commented 3 weeks ago

hi, I am new to Chronos.

How do I evaluate the performance of forecast ? is there a way to do BACKTESTING ?

Do we use MAE ?

lostella commented 3 weeks ago

Hi @yeongnamtan, please check out the discussion in #75

abdulfatir commented 1 day ago

Update: We have just open-sourced the datasets used in the paper (thanks @shchur!). Please check the updated README. We have also released an evaluation script and backtest configs to compute the WQL and MASE numbers as reported in the paper. Please follow the instructions in this README to evaluate on the in-domain and zero-shot benchmarks.