search

Kyubyong / transformer

A TensorFlow Implementation of the Transformer: Attention Is All You Need
Apache License 2.0
4.25k stars 1.29k forks source link

how to understand decoder_inputs have ["<s>"] while y_hat, y or x don't have ["<s>"] #159

Closed bozhenhhu closed 4 years ago

bozhenhhu commented 4 years ago

In the decoder part,Query is the embedding of decoder_inputs, while x and enc don't have [""], in the model.py eval parts, every circulation _decoder_inputs = tf.concat((yt_decoder_inputs, y_hat), 1),,, why the predicted y_hat doesn't have [""] and need to add this token over and over?

bozhenhhu commented 4 years ago

In the decoder part,Query is the embedding of decoder_inputs, while x and enc don't have [""], in the model.py eval parts, every circulation _decoder_inputs = tf.concat((yt_decoder_inputs, y_hat), 1)…… why the predicted y_hat doesn't have [""] and need to add this token over and over?

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.