Using Readout GRU with TimeDistributeDense

[NickShahML]

The Readout GRU works well when you only have to predict one output.

However, if you are trying to predict multiple outputs (sequence to sequence), you need to use TimeDistributedDense

So my question is: what do you do with the input_shape parameter? This is currently my model: However, Keras asks for two dimensions for the input_shape parameter.

readout = Sequential()
readout.add(TimeDistributedDense(y_maxlen, input_shape = (hidden_variables_decoding,), activation='softmax'))
gru_wr = GRUwithReadout(readout, return_sequences = True)

print '-------------------------------------Constructing Model ---------------------------------'
model = Sequential()
model.add(GRU(hidden_variables_encoding, input_shape=(x_maxlen, word2vec_dimension), return_sequences = False)) #note for the input shape that you did not put the number of samples 
# model.add(GRU(hidden_variables_encoding, return_sequences = False))
model.add(Dense(hidden_variables_encoding)) 
model.add(Activation('relu'))
model.add(RepeatVector(y_maxlen))
for z in range(0,number_of_decoding_layers):
    model.add(GRU(hidden_variables_decoding, return_sequences=True))
    model.add(Dropout(dropout))
model.add(gru_wr)
model.compile(loss='categorical_crossentropy', optimizer='adam')

Thanks alot for the help in advance!

[EderSantana]

Thanks for the feedback!!!

The readout will be used inside the RNN it should always be Dense based. Just make the RNN return_sequences=True instead.

[NickShahML]

My bad! This completely fixed it. I'm so excited to try this out! Thank you for pointing me towards this and making this.