maxjcohen / transformer

Implementation of Transformer model (originally from Attention is All You Need) applied to Time Series.

https://timeseriestransformer.readthedocs.io/en/latest/

GNU General Public License v3.0

842 stars 165 forks source link

Question about input of the decoder #55

Closed zlj-cs closed 2 years ago

zlj-cs commented 2 years ago

tst/transformer.py Line 138

Decoding stack

decoding = encoding

Add position encoding

if self._generate_PE is not None: positional_encoding = self._generate_PE(K, self._d_model) positional_encoding = positionalencoding.to(decoding.device) decoding.add(positional_encoding)

for layer in self.layers_decoding: decoding = layer(decoding, encoding)

Why are both input parameters of the decoder the output of the encoder? Shouldn't one of the inputs be a future sequence?

maxjcohen commented 2 years ago

Hi, please see #41 for a quick discussion on the subject.