tf.nn.sampled_softmax_loss API parameter error

[rexiaoyu]

It seems that the order of inputs and labels parameter is changed in the new tf.nn.sampled_softmax_loss API. But after changing the order, it still reports error: TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type int32 of argument 'x'.

CODE: modle.py `def _inference(self): print('Create inference')

If we use sampled softmax, we need an output projection.

    # Sampled softmax only makes sense if we sample less than vocabulary size.
    if config.NUM_SAMPLES > 0 and config.NUM_SAMPLES < config.DEC_VOCAB:
        w = tf.get_variable('proj_w', [config.HIDDEN_SIZE, config.DEC_VOCAB])
        b = tf.get_variable('proj_b', [config.DEC_VOCAB])
        self.output_projection = (w, b)

    def sampled_loss(inputs, labels):
        labels = tf.reshape(labels, [-1, 1])
        #return tf.nn.sampled_softmax_loss(tf.transpose(w), b, inputs, labels, 
        return tf.nn.sampled_softmax_loss(tf.transpose(w), b, labels, inputs,
                                          config.NUM_SAMPLES, config.DEC_VOCAB)
    self.softmax_loss_function = sampled_loss

    #single_cell = tf.nn.rnn_cell.GRUCell(config.HIDDEN_SIZE)
    #self.cell = tf.nn.rnn_cell.MultiRNNCell([single_cell] * config.NUM_LAYERS)
    single_cell = tf.contrib.rnn.GRUCell(config.HIDDEN_SIZE)
    self.cell = tf.contrib.rnn.MultiRNNCell([single_cell] * config.NUM_LAYERS)`

ERROR REPORT: Traceback (most recent call last): File "chatbot.py", line 261, in main() File "chatbot.py", line 258, in main chat() File "chatbot.py", line 203, in chat model.build_graph() File "D:\chatbot\tf-stanford-tutorials-master\tf-stanford-tutorials-master\assignments\chatbot\model.py", line 142, in build_graph self._create_loss() File "D:\chatbot\tf-stanford-tutorials-master\tf-stanford-tutorials-master\assignments\chatbot\model.py", line 94, in _create_loss softmax_loss_function=self.softmax_loss_function) File "D:\Python3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1195, in model_with_buckets softmax_loss_function=softmax_loss_function)) File "D:\Python3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1110, in sequence_loss softmax_loss_function=softmax_loss_function)) File "D:\Python3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1067, in sequence_loss_by_example crossent = softmax_loss_function(target, logit) File "D:\chatbot\tf-stanford-tutorials-master\tf-stanford-tutorials-master\assignments\chatbot\model.py", line 64, in sampled_loss config.NUM_SAMPLES, config.DEC_VOCAB) File "D:\Python3\lib\site-packages\tensorflow\python\ops\nn_impl.py", line 1189, in sampled_softmax_loss name=name) File "D:\Python3\lib\site-packages\tensorflow\python\ops\nn_impl.py", line 972, in _compute_sampled_logits array_ops.reshape(true_w, new_true_w_shape)) File "D:\Python3\lib\site-packages\tensorflow\python\ops\math_ops.py", line 267, in multiply return gen_math_ops._mul(x, y, name) File "D:\Python3\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 1625, in _mul result = _op_def_lib.apply_op("Mul", x=x, y=y, name=name) File "D:\Python3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 522, in apply_op inferred_from[input_arg.type_attr])) TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type int32 of argument 'x'.

[ade1963]

Try such code: ` if config.NUM_SAMPLES > 0 and config.NUM_SAMPLES < config.DEC_VOCAB: w_t = tf.get_variable('proj_w', [config.DEC_VOCAB, config.HIDDEN_SIZE]) w = tf.transpose(w_t) b = tf.get_variable('proj_b', [config.DEC_VOCAB]) self.output_projection = (w, b)

    def sampled_loss(labels, logits):
      labels = tf.reshape(labels, [-1, 1])
      local_w_t = tf.cast(w_t, tf.float32)
      local_b = tf.cast(b, tf.float32)
      local_inputs = tf.cast(logits, tf.float32)
      return tf.nn.sampled_softmax_loss(
               weights=local_w_t,
               biases=local_b,
               labels=labels,
               inputs=local_inputs,
               num_sampled=config.NUM_SAMPLES,
               num_classes=config.DEC_VOCAB)
    self.softmax_loss_function = sampled_loss

`

[tongda]

In the new version of model_with_bucket, the signature of argument softmax_loss_function has been changed:

softmax_loss_function: Function (labels-batch, inputs-batch) -> loss-batch to be used instead of the standard softmax (the default if this is None).

So you have to change the order of sampled_loss arguments to def sampled_loss(labels, inputs):

[crazyofapple]

@ade1963 after change the code, I get this. Traceback (most recent call last): File "chatbot.py", line 261, in <module> main() File "chatbot.py", line 256, in main train() File "chatbot.py", line 137, in train model.build_graph() File "/home/ngly/tf-stanford-tutorials/assignments/chatbot/model.py", line 143, in build_graph self._create_loss() File "/home/ngly/tf-stanford-tutorials/assignments/chatbot/model.py", line 111, in _create_loss softmax_loss_function=self.softmax_loss_function) File "/home/ngly/ngly/local/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1206, in model_with_buckets decoder_inputs[:bucket[1]]) File "/home/ngly/tf-stanford-tutorials/assignments/chatbot/model.py", line 86, in _seq2seq_f feed_previous=False) File "/home/ngly/ngly/local/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 848, in embedding_attention_seq2seq encoder_cell = copy.deepcopy(cell) File "/usr/lib/python2.7/copy.py", line 174, in deepcopy y = copier(memo) File "/home/ngly/ngly/local/lib/python2.7/site-packages/tensorflow/python/layers/base.py", line 476, in __deepcopy__ setattr(result, k, copy.deepcopy(v, memo)) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 230, in _deepcopy_list y.append(deepcopy(a, memo)) File "/usr/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 230, in _deepcopy_list y.append(deepcopy(a, memo)) File "/usr/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 230, in _deepcopy_list y.append(deepcopy(a, memo)) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 237, in _deepcopy_tuple y.append(deepcopy(a, memo)) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python2.7/copy.py", line 163, in deepcopy y = copier(x, memo) File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.7/copy.py", line 190, in deepcopy y = _reconstruct(x, rv, 1, memo) File "/usr/lib/python2.7/copy.py", line 343, in _reconstruct y.__dict__.update(state) AttributeError: 'NoneType' object has no attribute 'update' how to fix it?

[chiphuyen]

chatbot currently incompatible with tf 1. will update it soon.

[Pankajchandan]

Initialize new model Create placeholders Create inference Creating loss... It might take a couple of minutes depending on how many buckets you have. Traceback (most recent call last): File "chatbot.py", line 262, in <module> main() File "chatbot.py", line 257, in main train() File "chatbot.py", line 138, in train model.build_graph() File "/home/pankaj/git/s2s/model.py", line 133, in build_graph self._create_loss() File "/home/pankaj/git/s2s/model.py", line 101, in _create_loss softmax_loss_function=self.softmax_loss_function) File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1221, in model_with_buckets softmax_loss_function=softmax_loss_function)) File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1134, in sequence_loss softmax_loss_function=softmax_loss_function)) File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1089, in sequence_loss_by_example crossent = softmax_loss_function(labels=target, logits=logit) TypeError: sampled_loss() got an unexpected keyword argument 'logits'

This is the error i get while building the graph