seq2seq teacher forcing on tensorflow

[makseq]

I try to run test.py/seq2seq with teacher forcing on tensorflow backend:

if __name__ == '__main__':
    x = np.random.random((samples, input_length, input_dim))
    y = np.random.random((samples, output_length, output_dim))

    model = Seq2Seq(output_dim=output_dim, hidden_dim=hidden_dim, output_length=output_length,
                    input_shape=(input_length, input_dim), peek=True, depth=2, teacher_force=True)
    model.compile(loss='mse', optimizer='sgd')
    model.fit([x, y], y, epochs=1)

And I have this error:

Using TensorFlow backend.
Traceback (most recent call last):
  File "test.py", line 67, in <module>
    input_shape=(input_length, input_dim), peek=True, depth=2, teacher_force=True)
  File "/mnt/projects/quotes/research/seq2seq/seq2seq/models.py", line 201, in Seq2Seq
    initial_readout=encoded_seq, initial_state=states)
  File "/mnt/projects/quotes/research/recurrentshop/recurrentshop/engine.py", line 488, in __call__
    output = self.call(inputs, **kwargs)
  File "/mnt/projects/quotes/research/recurrentshop/recurrentshop/engine.py", line 594, in call
    input_length=input_length)
  File "/mnt/projects/quotes/research/recurrentshop/recurrentshop/backend/__init__.py", line 6, in <lambda>
    rnn = lambda *args, **kwargs: K.rnn(*args, **kwargs) + ([],)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 2641, in rnn
    swap_memory=True)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 2816, in while_loop
    result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 2640, in BuildLoop
    pred, body, original_loop_vars, loop_vars, shape_invariants)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 2590, in _BuildLoop
    body_result = body(*packed_vars_for_body)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 2627, in _step
    tuple(constants))
  File "/mnt/projects/quotes/research/recurrentshop/recurrentshop/engine.py", line 657, in step
    zero = K.cast(K.zeros((1,))[0], 'int32')
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 676, in zeros
    dtype, name)
  File "/usr/local/lib/python2.7/dist-packages/keras/backend/tensorflow_backend.py", line 380, in variable
    v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variables.py", line 213, in __init__
    constraint=constraint)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variables.py", line 322, in _init_from_args
    "initializer." % name)
ValueError: Initializer for variable recurrent_sequential_2/while/Variable/ is from inside a control-flow construct, such as a loop or conditional. When creating a variable inside a loop or conditional, use a lambda as the initializer.

Any ideas how to fix it?

[Cyberdog52]

I had a similar issue using the recurrentshop package of the same author. I 'fixed' it by setting the keras backend to theano. Hope this helps! Tell me if it worked :)

[makseq]

Yes, theano works for me. But this lib (seq2seq) has a lot of troubles and I decide to use pure tensorflow instead.

[yizhiql]

Add 'unroll=True' on tensorflow backend