which tensorflow version are you using? Type error when implement with tf v1.0

[EmbraceLife]

Hi, Thanks for this wonderful repo!

when I execute the line

outputs, states = tf.nn.seq2seq.embedding_attention_seq2seq(
                                            encoder_inputs,
                                            decoder_inputs,
                                            tf.nn.rnn_cell.BasicLSTMCell(size),
                                            num_encoder_symbols = en_vocab_size,
                                            num_decoder_symbols = de_vocab_size,
                                            embedding_size = 100,
                                            feed_previous = False,
                                            output_projection = output_projection,
                                            dtype = tf.float32)

it appear on my tensorflow v1.0 there is no such module, nor did v1.0 API mention it online.

AttributeError: "'module' object has no attribute 'seq2seq'"

Which tensorflow version are you using? How can I use the same version?

I tried to implement your code with tensorlfow v1.0 with some changes on the function names. I used those functions below:

tf.contrib.legacy_seq2seq.embedding_attention_seq2seq and tf.contrib.legacy_seq2seq.sequence_loss to replace tf.nn.seq2seq.embedding_attention_seq2seq and tf.nn.seq2seq.sequence_loss

However, when I execute loss = tf.contrib.legacy_seq2seq.sequence_loss(outputs, targets, target_weights, softmax_loss_function = sampled_loss), I got the following error:

Traceback (most recent call last):
  File "neural_machine_translation.py", line 120, in <module>
    loss = tf.contrib.legacy_seq2seq.sequence_loss(outputs, targets, target_weights, softmax_loss_function = sampled_loss)
  File "/Users/Natsume/miniconda2/envs/ml/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1110, in sequence_loss
    softmax_loss_function=softmax_loss_function))
  File "/Users/Natsume/miniconda2/envs/ml/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1067, in sequence_loss_by_example
    crossent = softmax_loss_function(target, logit)
  File "neural_machine_translation.py", line 116, in sampled_loss
    num_classes = de_vocab_size)
  File "/Users/Natsume/miniconda2/envs/ml/lib/python2.7/site-packages/tensorflow/python/ops/nn_impl.py", line 1191, in sampled_softmax_loss
    name=name)
  File "/Users/Natsume/miniconda2/envs/ml/lib/python2.7/site-packages/tensorflow/python/ops/nn_impl.py", line 974, in _compute_sampled_logits
    array_ops.reshape(true_w, new_true_w_shape))
  File "/Users/Natsume/miniconda2/envs/ml/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 357, in multiply
    return gen_math_ops._mul(x, y, name)
  File "/Users/Natsume/miniconda2/envs/ml/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 1625, in _mul
    result = _op_def_lib.apply_op("Mul", x=x, y=y, name=name)
  File "/Users/Natsume/miniconda2/envs/ml/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 522, in apply_op
    inferred_from[input_arg.type_attr]))
TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type int32 of argument 'x'.

Could you help me to fix this error? Thanks

[jsdtwry]

Same problem I have met. And I remove the softmax_loss definition and use default version: loss = tf.contrib.legacy_seq2seq.sequence_loss(outputs, targets, target_weights) It could run but get nan loss value... Could you give me some ideas? @Nemzy

[Nemzy]

Problem is with broken backward compatibilities between two different versions of TensorFlow. Expect fix soon, the code will be updated so it can work with the newest version of tf. @EmbraceLife @jsdtwry

[EmbraceLife]

Hi @Nemzy, are you waiting for a new version of tf before you can update your code? If so, that can be a while right?

I wonder which previous version did you use to make your notebook work, given v1.0 is not working for the moment. I want to try it with the older version, if it can work. Or do you have to tweak even that tf version in order to make the notebook to work? If so, could you tell me how?

Thanks

[Nemzy]

@EmbraceLife Just use these functions tf.contrib.legacy_seq2seq.embedding_attention_seq2seq and tf.contrib.legacy_seq2seq.sequence_loss and change sampled_loss(logits, labels) to sampled_loss(labels, logits) (this was changed in v1.0). I think it will work with these changes. I've used v0.12.

[EmbraceLife]

Thanks @Nemzy The previous error is gone, but I got a warning and an error: Inside the training section:

print '------------------TRAINING------------------'

set_trace()
with tf.Session() as sess:
    sess.run(init)

    t = time.time()
    for step in range(steps):
        feed = feed_dict(X_train, Y_train)

        backward_step(sess, feed)

Warning:

-> feed = feed_dict(X_train, Y_train)
(Pdb)
/Users/Natsume/miniconda2/envs/ml/lib/python2.7/site-packages/numpy/core/numeric.py:301: FutureWarning: in the future, full([64], 3) will return an array of dtype('int64')
  format(shape, fill_value, array(fill_value).dtype), FutureWarning)

Error:

-> backward_step(sess, feed)
(Pdb) n
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 15256 which is outside the valid range of [0, 512).  Label values: 203 1155 30 25 63 4 3979 51 8500 35 1210 127 7 3275 37 1769 30 37 9 819 22 4 202 22 7 19 7 168 49 1645 7 494 1636 329 7 3429 15256 3931 22 554 112 35 44 37 7 22 19 4 46 12256 46 4256 1282 203 22 569 135 4 4 49 4 11095 4 139
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 28883 which is outside the valid range of [0, 512).  Label values: 2229 10192 719 24177 29 22 185 309 23 5 918 581 59 5341 10 206 3610 12 86 7821 769 320 170 64 28883 85 95 506 3493 1191 1151 1656 13 1654 85 45 25465 5 409 7699 8025 32 67 14 53 178 5 16640 95 254 436 21 2 553 64 162 260 106 51 8124 25 167 583 43
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 27049 which is outside the valid range of [0, 512).  Label values: 4635 118 60 10 18 357 153 2 7108 978 11 10 15 12 1795 10 60 137 115 15380 242 6 2922 15 20 8 116 5 11 16 208 12 11 10 132 20 2 196 26 356 12 6545 52 599 27049 31 80 2 175 2 310 2905 3 2 52 11 2 6 201 2680 2 79 2 11
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 19410 which is outside the valid range of [0, 512).  Label values: 1117 50 6 107 15327 75 1602 3 36 2 3392 255 215 13567 52 2 29 8 93 13 283 2 145 1866 26 2 1551 5063 8 68 81 5 15488 24 377 24 3 2 123 19410 1283 109 6 6583 2 83 18 3 4074 3 31 55 3 3 75 107 3 2 2427 648 3 15 3 5549
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 27512 which is outside the valid range of [0, 512).  Label values: 2 1970 3718 2 19710 2 2 3 695 3 21457 2 2 2 2 3 15 6 3252 28 144 3 812 2 9317 3 2 11692 608 5 10 15 825 27512 812 258 3 3 252 17 23523 4713 358 39 3 6 7247 3 2 3 29 18 3 3 412 2 3 3 2 3260 3 16 3 13
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 7509 which is outside the valid range of [0, 512).  Label values: 3 455 2 3 2 3 3 3 13 3 232 3 3 3 3 3 138 7509 68 669 190 3 2 3 2 3 3 2 47 6 16 2 58 2 2 1241 3 3 111 610 93 70 2 185 3 6814 113 3 3 3 11 1270 3 3 653 3 3 3 3 69 3 68 3 0
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 15577 which is outside the valid range of [0, 512).  Label values: 3 8 3 3 3 3 3 3 3384 3 2 3 3 3 3 3 361 2 2 9 42 3 3 3 3 3 3 3 54 1051 11983 3 15577 3 3 87 3 3 2 2 13 1809 3 159 3 244 14078 3 3 3 833 14 3 3 2 3 3 3 3 2 3 2 3 66
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 14156 which is outside the valid range of [0, 512).  Label values: 3 83 3 3 3 3 3 3 2 3 3 3 3 3 3 3 40 3 3 40 2 3 3 3 3 3 3 3 6 2 14156 3 784 3 3 250 3 3 3 3 1091 2 3 530 3 34 2 3 3 3 2 5836 3 3 3 3 3 3 3 3 3 3 3 17
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 2587 which is outside the valid range of [0, 512).  Label values: 3 467 3 3 3 3 3 3 3 3 3 3 3 3 3 3 60 3 3 18 3 3 3 3 3 3 3 3 78 3 2 3 2587 3 3 811 3 3 3 3 84 3 3 47 3 20 3 3 3 3 3 13 3 3 3 3 3 3 3 3 3 3 3 14
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 20064 which is outside the valid range of [0, 512).  Label values: 3 3198 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 3 3 7903 3 3 3 3 3 3 3 3 198 3 3 3 2 3 3 2 3 3 3 3 481 3 3 2 3 276 3 3 3 3 3 10478 3 3 3 3 3 3 3 3 3 3 3 20064
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 6796 which is outside the valid range of [0, 512).  Label values: 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 3 3 3 3 3 3 3 3 23 3 3 3 3 3 3 3 3 3 3 3 6796 3 3 3 3 1658 3 3 3 3 3 2 3 3 3 3 3 3 3 3 3 3 3 5337
W tensorflow/core/framework/op_kernel.cc:993] Invalid argument: Received a label value of 7562 which is outside the valid range of [0, 512).  Label values: 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 7562 3 3 3 3 3 3 3 3 3 3 3 47 3 3 3 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2
InvalidArgumentError: InvalidA...ntError()

[Nemzy]

I am going to update my notebook today with tf v1.0. @EmbraceLife @jsdtwry

[EmbraceLife]

looking forward to it

[Nemzy]

Notebook has been updated (TensorFlow version r1.0). If you find any errors feel free to open new issue. I'm sorry to have kept you waiting, best regards. @EmbraceLife @jsdtwry

[EmbraceLife]

thanks, it is training without having a error, given checkpoints/ is provided before training.