search

Nixtla / utilsforecast

https://nixtlaverse.nixtla.io/utilsforecast
Apache License 2.0
42 stars 7 forks source link

Overall Performance Metric #32

Closed baniasbaabe closed 11 months ago

baniasbaabe commented 11 months ago

I see that the metrics are usually performed for every unique_id but is there a possibility to output an overall metric for all unique_ids?

jmoralez commented 11 months ago

Hey @baniasbaabe, thanks for raising this. Do you mean aggregate by unique_id first and then overall or just overall from the start?

baniasbaabe commented 11 months ago

Yes, just overall from start, so it shouldn‘t perform a groupby. I guess it‘s not possible rn, or?

jmoralez commented 11 months ago

It's usually not a good idea because the series can have different scales, so big errors in series with small values wouldn't produce a high RMSE for example. Also longer series would weigh more doing that. It's probably better to average the values across all ids, that way at least each serie weighs the same.