Open fujimotomh opened 7 years ago
Hi, I was wondering if someone could confirm my suspicion. I think this code in model.py is not ever used with the way sampling is done currently.
def loop(prev, _): prev = tf.matmul(prev, softmax_w) + softmax_b prev_symbol = tf.stop_gradient(tf.argmax(prev, 1)) return tf.nn.embedding_lookup(embedding, prev_symbol) outputs, last_state = seq2seq.rnn_decoder(inputs, self.initial_state, cell, loop_function=loop if infer else None, scope='rnnlm')
When I change to this, training and sampling seems to work fine
# def loop(prev, _): # prev = tf.matmul(prev, softmax_w) + softmax_b # prev_symbol = tf.stop_gradient(tf.argmax(prev, 1)) # return tf.nn.embedding_lookup(embedding, prev_symbol) outputs, last_state = seq2seq.rnn_decoder(inputs, self.initial_state, cell, scope='rnnlm')
Looking at the source for seq2seq.rnn_decoder, if input has length 1 (which it does when infer == True), the loop function is never used. Am I missing something? It almost looks like this code could replicate this paper.
@fujimotomh i agree with you ,i think the loop_function is not necessary,it has never been used
Hi, I was wondering if someone could confirm my suspicion. I think this code in model.py is not ever used with the way sampling is done currently.
When I change to this, training and sampling seems to work fine
Looking at the source for seq2seq.rnn_decoder, if input has length 1 (which it does when infer == True), the loop function is never used. Am I missing something? It almost looks like this code could replicate this paper.