Reshape Layer Problem

[sameerkhurana10]

Trying to Reshape the Output from the Embedding Layer but unsuccessful. below is the code:

main_input = Input(shape=self.input_shape)

embedded = Embedding(self.alphabet_size, self.embedding_dim, dropout = self.em_drop) (main_input)

print(embedded.get_shape()) # (?, 80, 2, 50)

# Want to convert the input from previous step to (?,80,100) but get error at this step!
#Basically 80 is the num of timesteps and each time step is the concatenation of two 50d continuous
#vectors i.e. each time step = 100d vector
embedded = Reshape((80, 100), input_shape = (80, 2, 50))(embedded)

print(embedded.get_shape())

lstm = LSTM(self.cell_dim, dropout_W = self.dropout_W, dropout_U = self.dropout_U)(embedded)

main_output = Dense(self.num_classes, activation='softmax')(lstm)

model = Model(input=main_input, output=main_output)

Error that I get at the Reshape step:

Traceback (most recent call last):
  File "./local/run_char_rnn.py", line 157, in <module>
    main(FLAGS)
  File "./local/run_char_rnn.py", line 92, in main
    model = rnn_obj.gen_model()
  File "/data/sls/qcri/asr/sameer/vardial/languageID/vardial_2017/models/lstm.py", line 44, in gen_model
    embedded = Reshape((80, 2, 50), input_shape = (80, 2, 50))(embedded)
  File "/data/sls/qcri/asr/sameer/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py", line 569, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/data/sls/qcri/asr/sameer/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py", line 632, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/data/sls/qcri/asr/sameer/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py", line 168, in create_node
    output_shapes = to_list(outbound_layer.get_output_shape_for(input_shapes[0]))
  File "/data/sls/qcri/asr/sameer/anaconda3/lib/python3.5/site-packages/keras/layers/core.py", line 336, in get_output_shape_for
    self.target_shape)
  File "/data/sls/qcri/asr/sameer/anaconda3/lib/python3.5/site-packages/keras/layers/core.py", line 330, in _fix_unknown_dimension
    raise ValueError(msg)
ValueError: total size of new array must be unchanged

What am I doing wrong?

[xtknight]

I have the same problem. I don't get it. I'm not changing the size of the array, but I still get this error. Logged BEFORE/AFTER are using get_shape()

Doing it with TensorFlow works. char_emb_reshape = tf.reshape(char_emb, [-1, 20, 15*100])

BEFORE char_emb (?, 20, 15, 100) AFTER char_emb (?, 20, 1500)

But with Keras, "total size of new array must be unchanged" error:

char_emb_reshape = Reshape((-1, 20, 15*100))(char_emb)

And apparently it wants me to change the '20' dimension to a '1', which makes the error go away but leaves me with an array that I don't know what to do with.

[suryasumukh]

The output dim of the embedding layer isn't right. I'm guessing this is what makes Reshape throw the error. model.output.shape and model.output_shape do not have the same dimension.

model = Sequential()
model.add(Embedding(257, 128, input_shape=(50, 5)))
print model.output_shape # => (None, 50, 128) should be (None, 50, 5, 128)
print model.ouput.shape # => TensorShape([Dimension(None), Dimension(50), Dimension(5), Dimension(128)])

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[greenli0511]

I have the same problem. As @suryasumukh said, "the output dim of the embedding layer isn't right".

model.add(Embedding(257, 128, input_shape=(50, 5)))
print model.output_shape # => (None, 50, 128) should be (None, 50, 5, 128)
print model.ouput.shape # => TensorShape([Dimension(None), Dimension(50), Dimension(5), Dimension(128)])
model.add(Reshape(50,2,64)) # this would work
model.add(Reshape(50,5*128,1)) # this would give the error "total size should not be changed"

However, although that reshape can work, it gives an error with "Incompatible shapes" at the end. I have no idea how to workaround. It seems that this may be a bug in keras.