search

llSourcell / tensorflow_chatbot

Tensorflow chatbot demo by @Sirajology on Youtube
1.45k stars 809 forks source link

Issues after changing all the imports for TF 1.0.0 #28

Open PeredurOmega opened 7 years ago

PeredurOmega commented 7 years ago

I'm trying to run this tensorflow_chatbot under the new version released TF 1.0.0. And I had a lot of "import troubles", which I fixed by following the advices of TF's devs.

tf.nn.rnn_cell. and most functions in tf.nn.rnn. (with the exception of dynamic_rnn and raw_rnn) are temporarily in tf.contrib.rnn. They will be moved back into core for TF 1.1.

But even after all this work I still have an error.

Traceback (most recent call last): File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 670, in _call_cpp_shape_fn_impl status) File "C:\Users\pauls\Anaconda3\lib\contextlib.py", line 66, in exit next(self.gen) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 469, in raise_exception_on_not_ok_status pywrap_tensorflow.TF_GetCode(status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Shape must be rank 2 but is rank 1 for 'model_with_buckets/sequence_loss/sequence_loss_by_example/sampled_softmax_loss/LogUniformCandidateSampler' (op: 'LogUniformCandidateSampler') with input shapes: [?]. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:/Users/pauls/Desktop/4. Stela/Chatbot project/chatBot-master/execute.py", line 303, in decode() File "C:/Users/pauls/Desktop/4. Stela/Chatbot project/chatBot-master/execute.py", line 192, in decode model = create_model(sess, True) File "C:/Users/pauls/Desktop/4. Stela/Chatbot project/chatBot-master/execute.py", line 95, in create_model model = seq2seq_model.Seq2SeqModel( gConfig['enc_vocab_size'], gConfig['dec_vocab_size'], _buckets, gConfig['layer_size'], gConfig['num_layers'], gConfig['max_gradient_norm'], gConfig['batch_size'], gConfig['learning_rate'], gConfig['learning_rate_decay_factor'], forward_only=forward_only) File "C:\Users\pauls\Desktop\4. Stela\Chatbot project\chatBot-master\seq2seq_model.py", line 135, in init softmax_loss_function=softmax_loss_function) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1195, in model_with_buckets softmax_loss_function=softmax_loss_function)) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1110, in sequence_loss softmax_loss_function=softmax_loss_function)) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1067, in sequence_loss_by_example crossent = softmax_loss_function(target, logit) File "C:\Users\pauls\Desktop\4. Stela\Chatbot project\chatBot-master\seq2seq_model.py", line 92, in sampled_loss self.target_vocab_size) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\ops\nn_impl.py", line 1191, in sampled_softmax_loss name=name) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\ops\nn_impl.py", line 947, in _compute_sampled_logits range_max=num_classes) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\ops\candidate_sampling_ops.py", line 134, in log_uniform_candidate_sampler seed2=seed2, name=name) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_candidate_sampling_ops.py", line 357, in _log_uniform_candidate_sampler name=name) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 763, in apply_op op_def=op_def) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2397, in create_op set_shapes_for_outputs(ret) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1757, in set_shapes_for_outputs shapes = shape_func(op) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1707, in call_with_requiring return call_cpp_shape_fn(op, require_shape_fn=True) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 610, in call_cpp_shape_fn debug_python_shape_fn, require_shape_fn) File "C:\Users\pauls\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 675, in _call_cpp_shape_fn_impl raise ValueError(err.message) ValueError: Shape must be rank 2 but is rank 1 for 'model_with_buckets/sequence_loss/sequence_loss_by_example/sampled_softmax_loss/LogUniformCandidateSampler' (op: 'LogUniformCandidateSampler') with input shapes: [?].

I would like to know if anyone has a solution?

Lezzio commented 7 years ago

Hey! I have exactly the same issue! Anyone ? Thanks!

yashkumar6640 commented 7 years ago

it is mainly due to the change in the order of arguments to a function(tf.nn.sampled_softmax). It is expecting inputs(rank 2) but you are passing labels(rank 1) So change the order of arguments according to definition of the function.

In tf 0.12.0: Definition : tf.nn.sampled_softmax_loss(weights, biases, inputs, labels, num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=True, partition_strategy="mod", name="sampled_softmax_loss")

In tf 1.0: tf.nn.sampled_softmax_loss(weights, biases, labels, inputs, num_sampled, num_classes, num_true=1, sampled_values=None, remove_accidental_hits=True, partition_strategy='mod', name='sampled_softmax_loss')

That is interchange "label" and "input" in you function arguments for function sampled_softmax_loss

vongruenigen commented 7 years ago

@yashkumar6640, I still get an error (with tf 1.0) when executing the following code and the error looks kinda similar:

logits = tf.transpose(self.decoder_logits_train, [1, 0, 2])
targets = tf.transpose(self.decoder_train_targets, [1, 0])
softmax_loss_fn = None

if self.__get_vocab_size() > self.cfg.get('max_vocabulary_size'):
    vocabulary_size = self.__get_vocab_size()
    num_hidden_units = self.cfg.get('num_hidden_units')
    num_sampled = self.cfg.get('sampled_softmax_number_of_samples')

    w = tf.get_variable('out_proj_w', [num_hidden_units, vocabulary_size], dtype=tf.float32)
    w_t = tf.transpose(w)
    b = tf.get_variable('out_proj_b', [vocabulary_size], dtype=tf.float32)

    self.output_projection = (w, b)

    def sampled_softmax_loss(labels, inputs):
        labels = tf.reshape(labels, [-1, 1])

        # We need to compute the sampled_softmax_loss using 32bit floats to
        # avoid numerical instabilities.
        local_w_t = tf.cast(w_t, tf.float32)
        local_b = tf.cast(b, tf.float32)
        local_inputs = tf.cast(inputs, tf.float32)

        return tf.nn.sampled_softmax_loss(
                local_w_t, local_b,
                labels, local_inputs,
                num_sampled, vocabulary_size)

    softmax_loss_fn = sampled_softmax_loss

self._loss = seq2seq.sequence_loss(logits=logits, targets=targets,
                                   weights=self.loss_weights,
                                   softmax_loss_function=softmax_loss_fn)

The error message which pops up seems to indicate that the problem still exists, even after chaning the position of the arguments:

ile "/Users/dvg/code/BA-ML-FS17/multimodel/seq2seq.py", line 68, in build
    self.__init_optimizer()
  File "/Users/dvg/code/BA-ML-FS17/multimodel/seq2seq.py", line 349, in __init_optimizer
    softmax_loss_function=softmax_loss_fn)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/loss.py", line 79, in sequence_loss
    crossent = softmax_loss_function(probs_flat, targets)
  File "/Users/dvg/code/BA-ML-FS17/multimodel/seq2seq.py", line 343, in sampled_softmax_loss
    num_sampled, vocabulary_size)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/ops/nn_impl.py", line 1191, in sampled_softmax_loss
    name=name)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/ops/nn_impl.py", line 995, in _compute_sampled_logits
    inputs, sampled_w, transpose_b=True) + sampled_b
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1855, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 1454, in _mat_mul
    transpose_b=transpose_b, name=name)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2397, in create_op
    set_shapes_for_outputs(ret)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1757, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1707, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
    debug_python_shape_fn, require_shape_fn)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 675, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Shape must be rank 2 but is rank 1 for 'sequence_loss/sampled_softmax_loss/MatMul_1' (op: 'MatMul') with input shapes: [?], [?,1000].

Do you have any clue on how to fix this problem?

AndyTsangChun commented 7 years ago

@yashkumar6640 Thx for your solution. But i encounter another problem, do you have any idea?

Traceback (most recent call last):
  File "execute.py", line 319, in <module>
    train()
  File "execute.py", line 137, in train
    model = create_model(sess, False)
  File "execute.py", line 104, in create_model
    model = seq2seq_model.Seq2SeqModel( gConfig['enc_vocab_size'], gConfig['dec_vocab_size'], _buckets, gConfig['layer_size'], gConfig['num_layers'], gConfig['max_gradient_norm'], gConfig['batch_size'], gConfig['learning_rate'], gConfig['learning_rate_decay_factor'], forward_only=forward_only)
  File "/Users/ATC/Desktop/TensorFlow/6. Chatbot/seq2seq_model.py", line 149, in __init__
    softmax_loss_function=softmax_loss_function)
  File "/Users/ATC/anaconda/envs/TF-NN/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1195, in model_with_buckets
    softmax_loss_function=softmax_loss_function))
  File "/Users/ATC/anaconda/envs/TF-NN/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1110, in sequence_loss
    softmax_loss_function=softmax_loss_function))
  File "/Users/ATC/anaconda/envs/TF-NN/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1067, in sequence_loss_by_example
    crossent = softmax_loss_function(target, logit)
  File "/Users/ATC/Desktop/TensorFlow/6. Chatbot/seq2seq_model.py", line 93, in sampled_loss
    self.target_vocab_size)
  File "/Users/ATC/anaconda/envs/TF-NN/lib/python2.7/site-packages/tensorflow/python/ops/nn_impl.py", line 1189, in sampled_softmax_loss
    name=name)
  File "/Users/ATC/anaconda/envs/TF-NN/lib/python2.7/site-packages/tensorflow/python/ops/nn_impl.py", line 972, in _compute_sampled_logits
    array_ops.reshape(true_w, new_true_w_shape))
  File "/Users/ATC/anaconda/envs/TF-NN/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 267, in multiply
    return gen_math_ops._mul(x, y, name)
  File "/Users/ATC/anaconda/envs/TF-NN/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 1625, in _mul
    result = _op_def_lib.apply_op("Mul", x=x, y=y, name=name)
  File "/Users/ATC/anaconda/envs/TF-NN/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 522, in apply_op
    inferred_from[input_arg.type_attr]))
TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type int32 of argument 'x'.

Update #1 : I found that, after switching labels and inputs for the ValueError in sampled_loss(). The labels tensor is float32 not int32, however, its a arbitrary tensor, I cannot cast the whole thing into int32 while I could neither iterate through to cast the element one by one. So I could I solve this?

@llSourcell Can I have some clues on solving this? Thx a lot.

fernlee commented 7 years ago

@AndyTsangChun In the functiondef sampled_loss(inputs, labels): you also need to change the order of the parameters to def sampled_loss(labels, inputs):

vishakhajadhav5 commented 4 years ago

@yashkumar6640, I still get an error (with tf 1.0) when executing the following code and the error looks kinda similar:

logits = tf.transpose(self.decoder_logits_train, [1, 0, 2])
targets = tf.transpose(self.decoder_train_targets, [1, 0])
softmax_loss_fn = None

if self.__get_vocab_size() > self.cfg.get('max_vocabulary_size'):
    vocabulary_size = self.__get_vocab_size()
    num_hidden_units = self.cfg.get('num_hidden_units')
    num_sampled = self.cfg.get('sampled_softmax_number_of_samples')

    w = tf.get_variable('out_proj_w', [num_hidden_units, vocabulary_size], dtype=tf.float32)
    w_t = tf.transpose(w)
    b = tf.get_variable('out_proj_b', [vocabulary_size], dtype=tf.float32)

    self.output_projection = (w, b)

    def sampled_softmax_loss(labels, inputs):
        labels = tf.reshape(labels, [-1, 1])

        # We need to compute the sampled_softmax_loss using 32bit floats to
        # avoid numerical instabilities.
        local_w_t = tf.cast(w_t, tf.float32)
        local_b = tf.cast(b, tf.float32)
        local_inputs = tf.cast(inputs, tf.float32)

        return tf.nn.sampled_softmax_loss(
                local_w_t, local_b,
                labels, local_inputs,
                num_sampled, vocabulary_size)

    softmax_loss_fn = sampled_softmax_loss

self._loss = seq2seq.sequence_loss(logits=logits, targets=targets,
                                   weights=self.loss_weights,
                                   softmax_loss_function=softmax_loss_fn)

The error message which pops up seems to indicate that the problem still exists, even after chaning the position of the arguments:

ile "/Users/dvg/code/BA-ML-FS17/multimodel/seq2seq.py", line 68, in build
    self.__init_optimizer()
  File "/Users/dvg/code/BA-ML-FS17/multimodel/seq2seq.py", line 349, in __init_optimizer
    softmax_loss_function=softmax_loss_fn)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/contrib/seq2seq/python/ops/loss.py", line 79, in sequence_loss
    crossent = softmax_loss_function(probs_flat, targets)
  File "/Users/dvg/code/BA-ML-FS17/multimodel/seq2seq.py", line 343, in sampled_softmax_loss
    num_sampled, vocabulary_size)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/ops/nn_impl.py", line 1191, in sampled_softmax_loss
    name=name)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/ops/nn_impl.py", line 995, in _compute_sampled_logits
    inputs, sampled_w, transpose_b=True) + sampled_b
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1855, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 1454, in _mat_mul
    transpose_b=transpose_b, name=name)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2397, in create_op
    set_shapes_for_outputs(ret)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1757, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1707, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
    debug_python_shape_fn, require_shape_fn)
  File "/Users/dvg/anaconda/envs/ba-ml-fs17/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 675, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Shape must be rank 2 but is rank 1 for 'sequence_loss/sampled_softmax_loss/MatMul_1' (op: 'MatMul') with input shapes: [?], [?,1000].

Do you have any clue on how to fix this problem?

vishakhajadhav5 commented 4 years ago

I got the same error as: ValueError: Dimensions must be equal, but are 256 and 24651 for 'model_with_buckets/sequence_loss/sequence_loss_by_example/sampled_softmax_loss/MatMul_1' (op: 'MatMul') with input shapes: [?,256], [?,24651]. can anyone help me?