thuml / Autoformer

About Code release for "Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting" (NeurIPS 2021), https://arxiv.org/abs/2106.13008
MIT License
2k stars 429 forks source link

Is trend_part need to do projecting when c_out not equals to d_in ? #83

Closed zhuwint closed 2 years ago

zhuwint commented 2 years ago

请教一下大佬,我有这么一个场景:

输入序列的维度是4,输出预测序列的维度是1(预测4维输入数据中的某一维)。

我发现在Autoformer中,对decoder输出的seasonal_part做了projection,而对trend_part没有做projection。因此最后输出的seasonal_parttrend_part的size不同,seasonal_part是[batch, len, 1]而trend_part是[batch, len, 4]。而Autoformer中最后是将seasonal_parttrend_part直接相加的,导致seasonal_part发生了广播。

https://github.com/thuml/Autoformer/blob/e116bbcf41f537f4ab53d172d9babfc0a026330f/layers/Autoformer_EncDec.py#L163-L173

https://github.com/thuml/Autoformer/blob/e116bbcf41f537f4ab53d172d9babfc0a026330f/models/Autoformer.py#L91-L99

如果我只想输出一维预测结果,是直接取trend_part对应的那一维的趋势(例如取trend_part[batch, len, 0:1])与seasonal_part相加,还是对trend_part也做一下projection

dec_out = trend_part[:, :, :c_out] + seasonal_part  
projection = nn.Linear(d_in, c_out)
trend_part = projection( trend_part)
dec_out = trend_part + seasonal_part 
wuhaixu2016 commented 2 years ago

您好,感谢关注! 不需要进行映射,其实只需要将trend的提取过程固定为其中一维即可。

zhuwint commented 2 years ago

非常感谢您的回复