search

philipperemy / keras-attention

Keras Attention Layer (Luong and Bahdanau scores).
Apache License 2.0
2.8k stars 675 forks source link

what do the h_t mean in the Attention model? #57

Closed fwyc0573 closed 2 years ago

fwyc0573 commented 3 years ago

Hi there! Thanks so much for implementing this and all of the other work that you do! I wanna know the meaning of h_t，i.e h_t = Lambda(lambda x: x[:, -1, :], output_shape=(hidden_size,), name='last_hidden_state')(hidden_states) . Well, in Luong's paper the h_t was used as the input the hidden state. But how to explain it in a scene which is not seq2seq?

philipperemy commented 2 years ago

@fwyc0573 try to refer to this issue: https://github.com/philipperemy/keras-attention-mechanism/issues/14.

Most of it is explained in here ;)