Last hidden states - Time series embeddings

time-series-foundation-models / lag-llama

Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting

Apache License 2.0

1.09k stars 121 forks source link

Last hidden states - Time series embeddings #31

Closed b7amine closed 2 months ago

b7amine commented 3 months ago

First , thank you for the great job , still did not finish the paper but trying to see what's the embedding space of time series looking like . It does not seem obvious how to get them out especially with the gluonTS wrapper , any thoughts here ? Thank you !

ashok-arjun commented 3 months ago

Hi, sorry, do you mean the embeddings of the model? You could extract the embedding from the model in this line. The embedding dimension in the current model is 144.

b7amine commented 3 months ago

Thanks , 2 few more questions as I did not dive deep in the code yet :

Does the model reinject the forecasts for the next steps , do the predictions become part of the context or not ?
Where in the code should I look/modify if I'd like to fine tune the model as I'm not able to reproduce some of the results on different time series data

ashok-arjun commented 3 months ago

Of course.

Yes, each forecast becomes part of the context as the timesteps progress.
Apologies for the delay - I was out on vacation so wasn't able to work on releasing the finetuning code. Would you mind waiting for 2 weeks for the release? Please let me know if you're using it for a deadline etc. and we can communicate through email (arjun.ashok AT servicenow.com).

Best, Arjun

ashok-arjun commented 2 months ago

Closing this issue as it is resolved. Feel free to open it again if required :)