search

yuqinie98 / PatchTST

An offical implementation of PatchTST: "A Time Series is Worth 64 Words: Long-term Forecasting with Transformers." (ICLR 2023) https://arxiv.org/abs/2211.14730
Apache License 2.0
1.37k stars 248 forks source link

Transformer Encoder #36

Closed Wty1122 closed 1 year ago

Wty1122 commented 1 year ago

Hi, it seems that only the encoder part of the transformer is used in the model. However, both Autoformer and FEDformer use the structure of encoder + decoder. Is it better to use the encoder than the full structure (encoder + decoder) on the time series forecasting task? Could you provide some literature or experimental support?

yuqinie98 commented 1 year ago

Hi! This is a very good questions. Some of the discussion about Transformer decoder may lead to performance degrading can been found in another recent paper (https://arxiv.org/pdf/2212.02789.pdf). But we haven't done serious analysis on that.