search

martin-gorner / tensorflow-rnn-shakespeare

Code from the "Tensorflow and deep learning - without a PhD, Part 2" session on Recurrent Neural Networks.
Apache License 2.0
534 stars 249 forks source link

Tensorflow 1.1 changes #8

Closed KonstantinKhabarlak closed 7 years ago

KonstantinKhabarlak commented 7 years ago

I've started using Tensorflow v1.1 RC and RNN learning fails with error.

Do not reuse GRU cells during training Fixes error: "Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.GRUCell object"

martin-gorner commented 7 years ago

ooh, good catch. I have to dig deeper to understand if the warning in TF 1.1 corresponds to a changed behaviour or to a but that I had in my code that is now tested for.

KonstantinKhabarlak commented 7 years ago

Here is full error message

"Attempt to reuse RNNCell %s with a different variable scope than its first use. First use of cell was with scope '%s', this attempt is with scope '%s'. Please create a new instance of the cell if you would like it to use a different set of weights. If before you were using: MultiRNNCell([%s(...)] * numlayers), change to: MultiRNNCell([%s(...) for in range(num_layers)]). If before you were using the same cell instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances (one for forward, one for reverse). In May 2017, we will start transitioning this cell's behavior to use existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation, so this error will remain until then.)"

Basically, I've followed what is proposed there

And the commit: https://github.com/tensorflow/tensorflow/commit/54d50ffec8df4f748694632dbe5ebde9971e2c9e

Hope it helps

martin-gorner commented 7 years ago

Your change looks good but I stil need to review wether it will make the saved weights incompatible, in which case I will have to regenerate all the data files used in rnn_play.py Thank you for your patience

martin-gorner commented 7 years ago

Thanks for you contribution. I have updated the code as you suggested. I also had to regenerate all the checkpoint files. All the TF 1.1 updates have been pushed.