can you explain more on dimension arguments to transformer class ?

maxjcohen / transformer

Implementation of Transformer model (originally from Attention is All You Need) applied to Time Series.

https://timeseriestransformer.readthedocs.io/en/latest/

GNU General Public License v3.0

842 stars 165 forks source link

can you explain more on dimension arguments to transformer class ? #46

Closed shivaniarbat closed 3 years ago

shivaniarbat commented 3 years ago

d_model here is output encoding dimension from the positional encoding. What is d_input and d_output ? shouldn't the value be 1 for both variables ?

d_input : Model input dimension. d_model : Dimension of the input vector. d_output: Model output dimension.

Can you explain this in relation to original paper ? (the application here to time-series )

maxjcohen commented 3 years ago

Hi, here we have

d_intput: dimension of the input vector. For NLP task, this would be the dimension of your embedding space. For time series, in the examples you can find in the documentation, I use about 30 variables as input.
d_model: dimension of the latent vector. This corresponds to the d_model in the original paper.
d_output: dimension of the output vector.