Tensorflow 1.1 RNNCell errors

[dresen]

I made a fresh install on a CentOS server, with python 2.7.13 (anaconda) and TF 1.1. python, TF and g2p-seq2seq are installed in userspace (with the --user flag)

Running python2 setup.py test gives the following errors (3 of the same):

(g2p) -bash-4.2$ python setup.py test running test running egg_info writing requirements to g2p_seq2seq.egg-info/requires.txt writing g2p_seq2seq.egg-info/PKG-INFO writing top-level names to g2p_seq2seq.egg-info/top_level.txt writing dependency_links to g2p_seq2seq.egg-info/dependency_links.txt writing entry points to g2p_seq2seq.egg-info/entry_points.txt reading manifest file 'g2p_seq2seq.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'g2p_seq2seq.egg-info/SOURCES.txt' running build_ext test_decode (tests.g2p_unittest.TestG2P) ... Loading vocabularies from tests/models/decode 2017-05-10 09:47:49.704509: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. 2017-05-10 09:47:49.704592: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. 2017-05-10 09:47:49.704616: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations. Creating 1 layers of 2 units. ERROR test_evaluate (tests.g2p_unittest.TestG2P) ... Loading vocabularies from tests/models/decode Creating 1 layers of 2 units. ERROR test_train (tests.g2p_unittest.TestG2P) ... Preparing G2P data Loading vocabularies from tests/models/train Reading development and training data. Creating model with parameters: Learning rate: 0.5 LR decay factor: 0.99 Max gradient norm: 5.0 Batch size: 64 Size of layer: 2 Number of layers: 1 Steps per checkpoint: 1 Max steps: 1 Optimizer: sgd

ERROR

====================================================================== ERROR: test_decode (tests.g2p_unittest.TestG2P)

Traceback (most recent call last): File "/home/akirkedal/software/g2p-seq2seq/tests/g2p_unittest.py", line 42, in test_decode g2p_model.load_decode_model() File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/g2p.py", line 96, in load_decode_model forward_only=True) File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/seq2seq_model.py", line 164, in init softmax_loss_function=softmax_loss_function) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1201, in model_with_buckets decoder_inputs[:bucket[1]]) File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/seq2seq_model.py", line 163, in self.target_weights, buckets, lambda x, y: seq2seq_f(x, y, True), File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/seq2seq_model.py", line 140, in seq2seq_f dtype=dtype) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 855, in embedding_attention_seq2seq encoder_cell, encoder_inputs, dtype=dtype) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn.py", line 197, in static_rnn (output, state) = call_cell() File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn.py", line 184, in callcell = lambda: cell(input, state) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 881, in call return self._cell(embedded, state) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 146, in call with _checked_scope(self, scope or "gru_cell", reuse=self._reuse): File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/contextlib.py", line 17, in enter return self.gen.next() File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 77, in _checked_scope type(cell).name)) ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.GRUCell object at 0x7f191c619950> with a different variable scope than its first use. First use of cell was with scope 'embedding_attention_seq2seq/embedding_attention_decoder/attention_decoder/gru_cell', this attempt is with scope 'embedding_attention_seq2seq/rnn/gru_cell'. Please create a new instance of the cell if you would like it to use a different set of weights. If before you were using: MultiRNNCell([GRUCell(...)] * numlayers), change to: MultiRNNCell([GRUCell(...) for in range(num_layers)]). If before you were using the same cell instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances (one for forward, one for reverse). In May 2017, we will start transitioning this cell's behavior to use existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation, so this error will remain until then.)

====================================================================== ERROR: test_evaluate (tests.g2p_unittest.TestG2P)

Traceback (most recent call last): File "/home/akirkedal/software/g2p-seq2seq/tests/g2p_unittest.py", line 30, in test_evaluate g2p_model.load_decode_model() File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/g2p.py", line 96, in load_decode_model forward_only=True) File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/seq2seq_model.py", line 164, in init softmax_loss_function=softmax_loss_function) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1201, in model_with_buckets decoder_inputs[:bucket[1]]) File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/seq2seq_model.py", line 163, in self.target_weights, buckets, lambda x, y: seq2seq_f(x, y, True), File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/seq2seq_model.py", line 140, in seq2seq_f dtype=dtype) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 855, in embedding_attention_seq2seq encoder_cell, encoder_inputs, dtype=dtype) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn.py", line 197, in static_rnn (output, state) = call_cell() File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn.py", line 184, in callcell = lambda: cell(input, state) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 881, in call return self._cell(embedded, state) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 146, in call with _checked_scope(self, scope or "gru_cell", reuse=self._reuse): File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/contextlib.py", line 17, in enter return self.gen.next() File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 77, in _checked_scope type(cell).name)) ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.GRUCell object at 0x7f18f8090d50> with a different variable scope than its first use. First use of cell was with scope 'embedding_attention_seq2seq/embedding_attention_decoder/attention_decoder/gru_cell', this attempt is with scope 'embedding_attention_seq2seq/rnn/gru_cell'. Please create a new instance of the cell if you would like it to use a different set of weights. If before you were using: MultiRNNCell([GRUCell(...)] * numlayers), change to: MultiRNNCell([GRUCell(...) for in range(num_layers)]). If before you were using the same cell instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances (one for forward, one for reverse). In May 2017, we will start transitioning this cell's behavior to use existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation, so this error will remain until then.)

====================================================================== ERROR: test_train (tests.g2p_unittest.TestG2P)

Traceback (most recent call last): File "/home/akirkedal/software/g2p-seq2seq/tests/g2p_unittest.py", line 22, in test_train g2p_model.create_train_model(g2p_params) File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/g2p.py", line 195, in create_train_model self.prepare_model(params) File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/g2p.py", line 167, in prepare_model optimizer=self.params.optimizer) File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/seq2seq_model.py", line 177, in init softmax_loss_function=softmax_loss_function) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1201, in model_with_buckets decoder_inputs[:bucket[1]]) File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/seq2seq_model.py", line 176, in lambda x, y: seq2seq_f(x, y, False), File "/home/akirkedal/software/g2p-seq2seq/g2p_seq2seq/seq2seq_model.py", line 140, in seq2seq_f dtype=dtype) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 855, in embedding_attention_seq2seq encoder_cell, encoder_inputs, dtype=dtype) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn.py", line 197, in static_rnn (output, state) = call_cell() File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn.py", line 184, in callcell = lambda: cell(input, state) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 881, in call return self._cell(embedded, state) File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 146, in call with _checked_scope(self, scope or "gru_cell", reuse=self._reuse): File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/contextlib.py", line 17, in enter return self.gen.next() File "/home/akirkedal/software/anaconda/envs/g2p/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 77, in _checked_scope type(cell).name)) ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.GRUCell object at 0x7f18d85bbdd0> with a different variable scope than its first use. First use of cell was with scope 'embedding_attention_seq2seq/embedding_attention_decoder/attention_decoder/gru_cell', this attempt is with scope 'embedding_attention_seq2seq/rnn/gru_cell'. Please create a new instance of the cell if you would like it to use a different set of weights. If before you were using: MultiRNNCell([GRUCell(...)] * numlayers), change to: MultiRNNCell([GRUCell(...) for in range(num_layers)]). If before you were using the same cell instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances (one for forward, one for reverse). In May 2017, we will start transitioning this cell's behavior to use existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation, so this error will remain until then.)

---

Ran 3 tests in 10.574s

FAILED (errors=3)

[dresen]

works with TF1.0.0

[nshmyrev]

Thank you for information. We are actually working on a port for https://github.com/google/seq2seq, hopefully it will be merged soon.

[jcbloch]

I'm having what looks like a similar problem on Mac OS X 10.12. 3 errors, all look like:

AttributeError: 'module' object has no attribute 'core_rnn_cell'

I installed tensor flow via vertualenv (see https://www.tensorflow.org/install/install_mac). Python 2.7.10, tensor flow 1.2.1

[I downgraded to tensor flow 1.0.1, and all tests now pass. This readme line isn't quite right: The tool requires TensorFlow at least version 1.0.0.]

[darkzbaron]

same issue

[aldanor]

TF is now at 1.3, wondering if there are any plans to support the newer API?

[nshmyrev]

No worries, guys. Two more weeks or so, we need to train good models.

[aldanor]

TensorFlow’s now at v1.4...

[jinserk]

Any progress? I saw the tf11 branch contains updated API, but there is no seq2seq directory holding tasks or models. So I got the error like:

Traceback (most recent call last):
  File "setup.py", line 48, in <module>
    test_suite = 'tests'
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/site-packages/setuptools/command/test.py", line 215, in run
    self.run_tests()
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/site-packages/setuptools/command/test.py", line 238, in run_tests
    **exit_kwarg
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/unittest/main.py", line 149, in parseArgs
    self.createTests()
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/unittest/main.py", line 158, in createTests
    self.module)
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/unittest/loader.py", line 130, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/home/jbaik/.pyenv/versions/2.7.14/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
  File "/d2/jbaik/g2p/g2p-seq2seq/tests/__init__.py", line 23, in <module>
    from tests import g2p_unittest
  File "/d2/jbaik/g2p/g2p-seq2seq/tests/g2p_unittest.py", line 4, in <module>
    from g2p_seq2seq import g2p
  File "/d2/jbaik/g2p/g2p-seq2seq/g2p_seq2seq/__init__.py", line 28, in <module>
    from g2p_seq2seq import g2p
  File "/d2/jbaik/g2p/g2p-seq2seq/g2p_seq2seq/g2p.py", line 45, in <module>
    from seq2seq import tasks, models
ImportError: No module named seq2seq

[joernweissenborn]

A short question: Does the problem lies in adapting to a refactored API or somewhere deeper?

In the former case, I could maybe help, if you give me some direction. In the latter, I am worried I can't, but maybe others, so it would still be worth to share some details about the problem.

[nurtas-m]

The problem in old seq2seq model that we adapted when creating g2p-seq2seq. Starting from new tf_ver1.x Tensorflow not supported that old seq2seq model. Instead of that Tensorflow added some modules which can be useful if you want to create your own seq2seq model: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/seq2seq. But I can't find good documentation and examples how to independently use this library without applying third-party projects.

There are two big projects which implement seq2seq: https://github.com/tensorflow/tensor2tensor and https://github.com/tensorflow/nmt. All the structure of seq2seq was completely changed compared with old seq2seq model. Tensor2tensor contains a lot of implementations for different seq2seq based tasks. And now we working on creating g2p-seq2seq based on tensor2tensor project.

[joernweissenborn]

Thanks for the update. Unfortunatly, this is out of my scope atm, but I wish you good progress! Thank you for your work!

[nurtas-m]

Thank you very much! I hope we will implement this new version soon.

[xushenkun]

@nurtas-m , Are the branch t2t and tf11 implemented completely? And where can we find the pretrained model? Thanks!

[nshmyrev]

@xushenkun the latest work is done here:

https://github.com/tensorflow/tensor2tensor/pull/466

The pretrained model is here:

http://alphacephei.com/kaldi/g2p-tensor2tensor.tar.gz

Not sure if you'll be able to run it quickly, maybe you can simply retrain yourself, it just takes a few hours.

[xushenkun]

@nshmyrev , Thanks!

[loretoparisi]

@nshmyrev @xushenkun since the PR has not been merged to TF master, what's the reference implementation at this point for this repo? Thanks!

By the way if I try to run the provided tensor2tensor model with TF 1.4.0 I get an error:

root@5f2793f9871c:~# g2p-seq2seq --interactive --model /models/g2p-tensor2tensor      
Traceback (most recent call last):
  File "/usr/local/bin/g2p-seq2seq", line 11, in <module>
    load_entry_point('g2p-seq2seq==5.0.0a0', 'console_scripts', 'g2p-seq2seq')()
  File "build/bdist.linux-x86_64/egg/g2p_seq2seq/app.py", line 92, in main
  File "build/bdist.linux-x86_64/egg/g2p_seq2seq/g2p.py", line 73, in load_decode_model
RuntimeError: Model not found in /models/g2p-tensor2tensor
root@5f2793f9871c:~# g2p-seq2seq --interactive --model /models/g2p-tensor2tensor/base/
Traceback (most recent call last):
  File "/usr/local/bin/g2p-seq2seq", line 11, in <module>
    load_entry_point('g2p-seq2seq==5.0.0a0', 'console_scripts', 'g2p-seq2seq')()
  File "build/bdist.linux-x86_64/egg/g2p_seq2seq/app.py", line 92, in main
  File "build/bdist.linux-x86_64/egg/g2p_seq2seq/g2p.py", line 77, in load_decode_model
  File "build/bdist.linux-x86_64/egg/g2p_seq2seq/data_utils.py", line 138, in load_params
UnboundLocalError: local variable 'num_layers' referenced before assignment

[nurtas-m]

Use the project on t2t branch. We still couldn't merge this branch with master, because of minor problems still exists.

[nurtas-m]

Closed, because it is not relevant anymore.