tensorflow / nmt

TensorFlow Neural Machine Translation Tutorial
Apache License 2.0
6.37k stars 1.96k forks source link

Error when building BasicDecoder with GreedyEmbeddingHelper #218

Closed mcemilg closed 6 years ago

mcemilg commented 6 years ago

I'm trying to build a basic encoder-decoder model. I build the model for training graph and it works perfect. The helper of the decoder is tf.contrib.seq2seq.TrainingHelper . But when I switch to helper to tf.contrib.seq2seq.GreedyEmbeddingHelper it throws a shape error.

Here it is my working helper.

helper = tf.contrib.seq2seq.TrainingHelper(
decoder_emb_inp, decoder_lengths, time_major=True)

And here it is the what I want to do.

start_tokens = tf.fill([batch_size], vezins_dict[start_token_str])
    end_token = vezins_dict[end_token_str]

helper = tf.contrib.seq2seq.GreedyEmbeddingHelper(decoder_emb_inp, 
    start_tokens, end_token)

I am using the same decoder and dynamic_decoding. It works with TrainingHelper, but it didn't work with GreedyEmbeddingHelper.

# decoder   
decoder = tf.contrib.seq2seq.BasicDecoder(
    decoder_cell, helper, encoder_state, 
    output_layer=projection_layer)

# Dynamic decoding
outputs, _, _ = tf.contrib.seq2seq.dynamic_decode(decoder)
logits = outputs.rnn_output

And here it is the error.

ValueError: linear is expecting 2D arguments: 
  [TensorShape([Dimension(20), Dimension(20), Dimension(10)]), 
  TensorShape([Dimension(20), Dimension(128)])]
ahmedhosny commented 6 years ago

Had the same error. Make sure you are passing the tf.nn.embedding_lookup to the inputs arg of the tf.contrib.seq2seq.TrainingHelper - and - passing the embedding weight matrix itself to embedding arg of tf.contrib.seq2seq.GreedyEmbeddingHelper. From your code, you are passing the same variable decoder_emb_inp to both.

mcemilg commented 6 years ago

Yeah actually I solved the issue like your said. And I forget to close the issue, thanks.