search

amazon-science / chronos-forecasting

Chronos: Pretrained (Language) Models for Probabilistic Time Series Forecasting
https://arxiv.org/abs/2403.07815
Apache License 2.0
2.02k stars 238 forks source link

Fine-tuning on a single time series #98

Closed teshnizi closed 2 weeks ago

teshnizi commented 3 weeks ago

Hello,

I only have a single time series, and I want to do forecasting on it. Does it make sense to do fine-tuning in this case?

I was thinking maybe I could split the data chronologically (use data from 2020 to 2022 for training and data from 2022 to 2024 for testing), but I'm not sure if that makes sense.

Thanks

lostella commented 3 weeks ago

@teshnizi I think fine-tuning on a single time series may not be ideal. First it's worth doing some backtesting on the series using the pretrained model, and see how accurate it appears to be on historical data. Did you try that already?

teshnizi commented 3 weeks ago

Thanks for your reply @lostella. I see. I did try it out-of-the-box, but the results I got after fine-tuning it on the first part of the time series were better (It went from like 20% to 10% error rate on the unseen data). It overfits when I tune it for more than a couple thousand iterations.

Our of curiosity, Have you guys thought about potential extensions to the model that would allow for the use of domain knowledge?

lostella commented 3 weeks ago

@teshnizi not concretely, I believe, but I think it's an interesting direction!

teshnizi commented 2 weeks ago

I'd love to have a chat and explore potential future directions if that is of interest to you.