graykode / nlp-tutorial

Natural Language Processing Tutorial for Deep Learning Researchers
https://www.reddit.com/r/MachineLearning/comments/amfinl/project_nlptutoral_repository_who_is_studying/
MIT License
14.07k stars 3.91k forks source link

Seq2Seq(Attention)Input Shape Question #31

Open dpyneo opened 5 years ago

dpyneo commented 5 years ago

Seq2Seq(Attention)\Seq2Seq(Attention)-Tensor.py

The shape of the input should be [max_time, batch_size,...]. The input = tf. transpose (dec_inputs, [1, 0, 2]) has already been transformed. In tf. expand_dims (inputs [i], 1), the expansion is indeed one dimension. It seems that there should be zero dimension expansion here. Although the final shape is correct, whether it is intentional or not is here. What about a little trick?

dpyneo commented 5 years ago

I would also like to ask if this is attention can be used for LSTM timing prediction. The main problem that bothers me is this n_class = len (word_dict). Can this be considered as a different feature of input? In the end, your project is very good. It has benefited a lot. It's very good. Thank you very much.

dpyneo commented 5 years ago

'S i want a beer', 'i want a beer E',Is this the last attention mechanism not aligned?

searchlink commented 5 years ago

Seq2Seq(Attention)\Seq2Seq(Attention)-Tensor.py

The shape of the input should be [max_time, batch_size,...]. The input = tf. transpose (dec_inputs, [1, 0, 2]) has already been transformed. In tf. expand_dims (inputs [i], 1), the expansion is indeed one dimension. It seems that there should be zero dimension expansion here. Although the final shape is correct, whether it is intentional or not is here. What about a little trick?

I have the same question. Do you solve it ?

Oh, I have solve it.You are right.However, batch_size=1,so it has no effect。

cqray1990 commented 4 years ago

i think the code is only for batch)size=1,isn't is ?