search

time-series-foundation-models / lag-llama

Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
Apache License 2.0
1.08k stars 121 forks source link

problem dataset weather and mae metric #60

Closed m6129 closed 3 weeks ago

m6129 commented 1 month ago

Hello, Dear Developer Can I know if you have performed any operations on dataset weather, before feeding it into the model, I am getting an error TypeError: object of type 'NoneType' has no len()

I also wanted to ask your advice, if it doesn't take a lot of your attention, if you could tell me how to calculate the MAE metric Thanks

ashok-arjun commented 3 weeks ago

Hi,

Is the problem with the weather dataset solved? I can look into the error if it's not resolved.

As for the MAE metric, we do not recommend it in fact, since we did not design our model with point-forecasting in mind. However, if you'd like to compute it for a paper/report, you could use the MAE value returned by the GluonTS Evaluator along with other metrics, or using the mean of the probabilistic forecast and then computing MAE, or using the median of the probabilistic forecast and then computing MAE (you can try the 3 methods and choose the one with the best average MAE, since we do not provide any "default" way to compute MAE/give point-predictions). I request that you indicate that our model was meant for probabilistic forecasting, in case you report any point-forecasting metric.