amazon-science / chronos-forecasting

Chronos: Pretrained (Language) Models for Probabilistic Time Series Forecasting
https://arxiv.org/abs/2403.07815
Apache License 2.0
2.03k stars 238 forks source link

What is the size of embeddings? #31

Open arittenbach opened 3 months ago

arittenbach commented 3 months ago

I'm interested in using embeddings generated by Chronos for training a downstream anomaly detection model. For these models, if I use your sample example for generation of embeddings, I get 144 embedding vectors, which is the same length of time series in the example you provide. However, with my test data case, I have a time series of length 300000, and when I run that through your model for embedding generation I end up with 512 embedding vectors. Is there an upper bound of time series length that I should be using with this model, or is this expected output? Thanks so much!

abdulfatir commented 3 months ago

@arittenbach The current Chronos models were trained with a context length of upto 512, so when you pass time series of length greater than that only the last 512 steps are actually used. See this line: https://github.com/amazon-science/chronos-forecasting/blob/main/src/chronos/chronos.py#L140

One potential way of extracting embeddings for a long time series is to feed sliding windows into the model.