Add some notes to these models, especially the corresponding paper links

[XYZOK000]

Could you add some notes to these models, especially the corresponding paper links? For example, in this paper, Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting, I don't know which model to use. Thank you. pytorch_model_dict = { "MultiAttnHeadSimple": MultiAttnHeadSimple, "SimpleTransformer": SimpleTransformer, "TransformerXL": TransformerXL, "DummyTorchModel": DummyTorchModel, "LSTM": LSTMForecast, "SimpleLinearModel": SimpleLinearModel, "CustomTransformerDecoder": CustomTransformerDecoder, "DARNN": DARNN, "DecoderTransformer": DecoderTransformer, "BasicAE": AE, "Informer": Informer }

[isaacmg]

Yeah this should probably be updated on the ReadMe as well but

Vanilla LSTM->LSTM: A basic LSTM that is suitable for multivariate time series forecasting and transfer learning. Full transformer -> SimpleTransformer: The full original transformer with all 8 encoder and decoder blocks. Requires passing the target in at inference. Simple Multi-Head Attention->MultiAttnHeadSimple: A simple multi-head attention block and linear embedding layers. Suitable for transfer learning. Transformer with a linear decoder->CustomTransformerDecoder: A transformer with n-encoder blocks (this is tunable) and a linear decoder. DA-RNN:->ARNN A well rounded model with which utilizes a LSTM + attention. Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting -> DecoderTransformer: Transformer XL (not fully supported yet): Informer -> "Informer": Beyond Efficient Transformer for Long Sequence Time-Series Forecasting DeepAR -> DeepAR (only available on certain branches"

So it is called DecoderTransformer in the model_dict

[XYZOK000]

Thank you for your reply. I have another question for you. In paper Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting, there is such a saying "We propose convolutional self-attention by employing causal convolutions to produce queries and keys in the self-attention layer." Is this the code "self.query_key = nn.Conv1d(n_embd, n_embd n_head 2, self.q_len).cuda()self.value = Conv1D(n_embd * n_head, 1, n_embd)"corresponding to "causal revolutions"?

Is this the code corresponding to "LogSparse Transformer"?

------------------ 原始邮件 ------------------ 发件人: "AIStream-Peelout/flow-forecast" @.>; 发送时间: 2021年11月13日(星期六) 凌晨2:29 @.>; @.**@.>; 主题: Re: [AIStream-Peelout/flow-forecast] Add some notes to these models, especially the corresponding paper links (Issue #443)

Yeah this should probably be updated on the ReadMe as well but Vanilla LSTM->LSTM: A basic LSTM that is suitable for multivariate time series forecasting and transfer learning. Full transformer: The full original transformer with all 8 encoder and decoder blocks. Requires passing the target in at inference. Simple Multi-Head Attention->MultiAttnHeadSimple: A simple multi-head attention block and linear embedding layers. Suitable for transfer learning. Transformer with a linear decoder->CustomTransformerDecoder: A transformer with n-encoder blocks (this is tunable) and a linear decoder. DA-RNN:->ARNN A well rounded model with which utilizes a LSTM + attention. Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting -> DecoderTransformer: Transformer XL (not fully supported yet): Informer -> "Informer": Beyond Efficient Transformer for Long Sequence Time-Series Forecasting DeepAR -> DeepAR (only available on certain branches"
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.