search

time-series-foundation-models / lag-llama

Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting
Apache License 2.0
1.2k stars 146 forks source link

First prediction value too far from the latest known value #48

Open evindor opened 5 months ago

evindor commented 5 months ago

Hello and thank you for the model. I'm experimenting with my dataset, and one kind of systematic error I found are charts like below, where the first prediction point is way off the latest known point from the context. I'm wondering what could be the reason for this and if the problem could be alleviated somehow? Many thanks.

Screenshot 2024-04-17 at 16 11 13 Screenshot 2024-04-17 at 16 10 29 Screenshot 2024-04-17 at 16 09 59 Screenshot 2024-04-17 at 16 09 32
ashok-arjun commented 5 months ago

Hi! That is weird indeed, are you doing zero-shot prediction? What is your context length and prediction length?

evindor commented 5 months ago

The images above are from 24-period prediction length and 4*24 context length, fine-tuned for 200 epochs on a dataset ~12,000 series. However, I saw similar behavior with zero-shot, and on varying context lengths with/without rope scaling. Interesting that sometimes the prediction is good, following the pattern, but it is offset for some reason.

wizardsd commented 3 months ago

I have the same problem, even without fine tuning. This increases CRPS and makes it difficult to compare performance across different context lengths.