search

awslabs / gluonts

Probabilistic time series modeling in Python
https://ts.gluon.ai
Apache License 2.0
4.41k stars 740 forks source link

SamFormer multivariate Estimator #3196

Open kashif opened 1 week ago

kashif commented 1 week ago

Add samformer model https://arxiv.org/abs/2402.10198

cc @romilbert

kashif commented 1 week ago

@romilbert this implementation differs, I believe, from yours in that I am using the mu and sigma from the instance normalization as inputs to the model as well. This serves the purpose of RevIN + non-stationary transformer and was first done in the DeepAR paper, which as far as I remember the RevIN or the SAMFromer paper does not mention....

Apart from that the output head is an independent distribution head instead of a point forecasting head...

romilbert commented 1 week ago

Hi @kashif, thanks for your interest in SAMformer. I appreciate your feedback. Incorporating mu and sigma from RevIN as additional inputs to the model is an interesting idea (similar to the DeepAR paper as you said, which we did reference in SAMformer paper). However, I have tried incorporating them into the transformer in different ways, and it did not improve our current results.

kashif commented 1 week ago

@romilbert that is because, I think, you are using globally standardized data while here the data continues to be in the original magnitude, and thus in your setup the use of the summary statistics do not help as they are more or less 0 and 1...