search

philipperemy / keras-attention

Keras Attention Layer (Luong and Bahdanau scores).
Apache License 2.0
2.8k stars 675 forks source link

Output with multiple time steps #58

Closed Kian-Soon closed 2 years ago

Kian-Soon commented 2 years ago

Hi,

Can this be used for predicting output with multiple time-steps? If no, how can the code be changed to accommodate this? Thanks.

philipperemy commented 2 years ago

@Kian-Soon yes you can predict on multiple time-steps. The output shape of the Attention layer is a 2D tensor with shape (batch_size, units). From there, you can use RepeatVector(n=5) if you want to output 5 time-steps. On top of that, use a TimeDistributed(Dense(1)) if each time-step is of output_dim=1

https://datascience.stackexchange.com/questions/46491/what-is-the-job-of-repeatvector-and-timedistributed