keras-team / keras

Deep Learning for humans
http://keras.io/
Apache License 2.0
61.97k stars 19.46k forks source link

Problem understanding implemented RNN architecture #9603

Closed fera0013 closed 3 years ago

fera0013 commented 6 years ago

I am trying to understand the architecture of the following simple RNN model

def simple_model(input_shape, output_sequence_length, english_vocab_size, french_vocab_size):
    input = Input(shape=input_shape[1:], name="input")
    rnn = LSTM(256, return_sequences=True,name="lstm_1")(input)
    output = Dense(french_vocab_size, activation='softmax',name="softmax_output")(rnn)
    model = Model(inputs=input, outputs=output)
    model.compile(loss=sparse_categorical_crossentropy,
                  optimizer=Adam(),
                  metrics=['accuracy'])
    return model

tmp_x = pad(preproc_english_sentences, max_french_sequence_length)
tmp_x = tmp_x.reshape((-1, preproc_french_sentences.shape[-2], 1))

# Train the neural network
simple_rnn_model = simple_model(
    tmp_x.shape,
    max_french_sequence_length,
    english_vocab_size,
    french_vocab_size)
print(simple_rnn_model.summary())

which is


Layer (type)                 Output Shape              Param #   
=================================================================
input (InputLayer)           (None, 21, 1)             0         
_________________________________________________________________
lstm_1 (LSTM)                (None, 21, 256)           264192    
_________________________________________________________________
softmax_output (Dense)       (None, 21, 344)           88408     
=================================================================
Total params: 352,600
Trainable params: 352,600
Non-trainable params: 0

Does that mean I have 21 timesteps and the input for each timestep is connected to 256 LSTMs, which are connected to 344 dense Neurons (in each time step) ?

abhijitghate commented 6 years ago

@fera0013 did you complete the project? I guess we are on the same page. I am having some different issue with the same project. Lemme know.