Closed ninesunqian closed 5 years ago
In model.py these are some lines
"self.batch_size = tf.size(self.iterator.source_sequence_length) " ..... "start_tokens = tf.fill([self.batch_size], tgt_sos_id)" .... " crossent = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=target_output, logits=logits) target_weights = tf.sequence_mask(self.iterator.target_sequence_length, max_time, dtype=logits.dtype) ..... loss = tf.reduce_sum(crossent * target_weights) / tf.to_float(self.batch_size) "
1 self.source_sequence_length is better than self.batch_size. 2 why not loss = tf.reduce_sum(crossent * target_weights) / target_sequence_length ?
Thanks!
In model.py these are some lines
"self.batch_size = tf.size(self.iterator.source_sequence_length) " ..... "start_tokens = tf.fill([self.batch_size], tgt_sos_id)" .... " crossent = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=target_output, logits=logits) target_weights = tf.sequence_mask(self.iterator.target_sequence_length, max_time, dtype=logits.dtype) ..... loss = tf.reduce_sum(crossent * target_weights) / tf.to_float(self.batch_size) "
1 self.source_sequence_length is better than self.batch_size. 2 why not loss = tf.reduce_sum(crossent * target_weights) / target_sequence_length ?
Thanks!