Tested with tensorflow?

[sushrutkarnik]

@braingineer

Hey thanks for implementing the CWRNN! I tried it out with the Theano backend and it worked really well, but when I tried it with the tensorflow backend, it fails with

ValueError: Expected concat_dim in range [0, 0), but got 1

Here's a small example (which is a part of a bigger network) which fails for me.

model = Sequential()
periods = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 16, 32, 64, 128, 256, 512]
model.add(ClockworkRNN(output_dim=len(periods), period_spec=periods, input_shape=(1000, 1)))

Do you see this too? I'm using Keras 1.1.0 and I tried this with tensorflow 0.8.0, 0.9.0, 1.0.0 and 1.1.0 and got similar errors in all of them.

Full trace:

ValueErrorTraceback (most recent call last)
<ipython-input-12-be0ebdd10677> in <module>()
      5 
      6 periods = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 16, 32, 64, 128, 256, 512]
----> 7 model.add(ClockworkRNN(output_dim=len(periods), period_spec=periods, input_shape=(1000, 1),  stateful = False))
      8 
      9 

/home/ubuntu/.local/lib/python2.7/site-packages/keras/models.pyc in add(self, layer)
    274                 else:
    275                     input_dtype = None
--> 276                 layer.create_input_layer(batch_input_shape, input_dtype)
    277 
    278             if len(layer.inbound_nodes) != 1:

/home/ubuntu/.local/lib/python2.7/site-packages/keras/engine/topology.pyc in create_input_layer(self, batch_input_shape, input_dtype, name)
    368         # and create the node connecting the current layer
    369         # to the input layer we just created.
--> 370         self(x)
    371 
    372     def assert_input_compatibility(self, input):

/home/ubuntu/.local/lib/python2.7/site-packages/keras/engine/topology.pyc in __call__(self, x, mask)
    512         if inbound_layers:
    513             # this will call layer.build() if necessary
--> 514             self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
    515             input_added = True
    516 

/home/ubuntu/.local/lib/python2.7/site-packages/keras/engine/topology.pyc in add_inbound_node(self, inbound_layers, node_indices, tensor_indices)
    570         # creating the node automatically updates self.inbound_nodes
    571         # as well as outbound_nodes on inbound layers.
--> 572         Node.create_node(self, inbound_layers, node_indices, tensor_indices)
    573 
    574     def get_output_shape_for(self, input_shape):

/home/ubuntu/.local/lib/python2.7/site-packages/keras/engine/topology.pyc in create_node(cls, outbound_layer, inbound_layers, node_indices, tensor_indices)
    147 
    148         if len(input_tensors) == 1:
--> 149             output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
    150             output_masks = to_list(outbound_layer.compute_mask(input_tensors[0], input_masks[0]))
    151             # TODO: try to auto-infer shape if exception is raised by get_output_shape_for

/home/ubuntu/.local/lib/python2.7/site-packages/keras/layers/recurrent.pyc in call(self, x, mask)
    213                                              constants=constants,
    214                                              unroll=self.unroll,
--> 215                                              input_length=input_shape[1])
    216         if self.stateful:
    217             self.updates = []

/home/ubuntu/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.pyc in rnn(step_function, inputs, initial_states, go_backwards, mask, constants, unroll, input_length)
   1177                 state = states[0]
   1178             else:
-> 1179                 state = tf.concat(1, states)
   1180 
   1181         if mask is not None:

/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.pyc in concat(concat_dim, values, name)
    695   return gen_array_ops._concat(concat_dim=concat_dim,
    696                                values=values,
--> 697                                name=name)
    698 
    699 

/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.pyc in _concat(concat_dim, values, name)
    412   """
    413   result = _op_def_lib.apply_op("Concat", concat_dim=concat_dim,
--> 414                                 values=values, name=name)
    415   return result
    416 

/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.pyc in apply_op(self, op_type_name, name, **keywords)
    701           op = g.create_op(op_type_name, inputs, output_types, name=scope,
    702                            input_types=input_types, attrs=attr_protos,
--> 703                            op_def=op_def)
    704           outputs = op.outputs
    705           return _Restructure(ops.convert_n_to_tensor(outputs),

/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.pyc in create_op(self, op_type, inputs, dtypes, input_types, name, attrs, op_def, compute_shapes, compute_device)
   2317                     original_op=self._default_original_op, op_def=op_def)
   2318     if compute_shapes:
-> 2319       set_shapes_for_outputs(ret)
   2320     self._add_op(ret)
   2321     self._record_op_seen_by_control_dependencies(ret)

/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.pyc in set_shapes_for_outputs(op)
   1709       raise RuntimeError("No shape function registered for standard op: %s"
   1710                          % op.type)
-> 1711   shapes = shape_func(op)
   1712   if shapes is None:
   1713     raise RuntimeError(

/home/ubuntu/.local/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.pyc in _ConcatShape(op)
    751       if value_shape.ndims is not None and concat_dim >= value_shape.ndims:
    752         raise ValueError("Expected concat_dim in range [0, %d), but got %d" %
--> 753                          (value_shape.ndims, concat_dim))
    754       before = output_shape[:concat_dim].merge_with(value_shape[:concat_dim])
    755       at = output_shape[concat_dim] + value_shape[concat_dim]

ValueError: Expected concat_dim in range [0, 0), but got 1

As far as I can tell this is caused by

            initial_states[-1] = K.variable(0.)

,in your get_initial_states function, K.variable(0.) not having the right shape for tf.concat. Not sure why theano works fine though..

[braingineer]

@sushrutkarnik Sorry for the lack of response. Many things have eaten up my time.

I'm installing tensorflow now and will be debugging this. I hadn't really bothered to learn tensorflow because the switch cost for my work was too high, but this seems like a good opportunity to dive into it.