Input sizing for BLSTM

[justiceamoh]

thanks for your keras blstm layer implementation; has been very helpful. I am having issues with sizing my inputs for training the blstm. I create my stack of layers with:

NUM_FEATURES  = 64   
MAX_LEN       = 16      
NUM_CLASSES   = 2   

model = Sequential()
# model.add(Transform((NUM_FEATURES,)))
model.add(BiDirectionLSTM(NUM_FEATURES, 128, output_mode='sum'))
model.add(BiDirectionLSTM(128,128, output_mode='sum'))
model.add(Dense(128,1,activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam', class_mode="binary")
model.fit(X_train, y_train, batch_size=20, nb_epoch=3, validation_data=(X_valid, y_valid))

And while the model is able to compile, it throws an error when attempting to fit:

TypeError: ('Bad input argument to theano function with name "build/bdist.macosx-10.5-x86_64/egg/keras/models.py:127"  at index 1(0-based)', 
'Wrong number of dimensions: expected 3, got 2 with shape (20, 1).')

the sizes of my training and validation data are:

5766 train sequences
3072 test sequences
X_train shape: (5766, 16, 64)
X_valid shape: (1441, 16, 64)

I know you added the Transform method to help size inputs appropriately for the blstm. However, since my input data is already 3D and since I have no mlp before the blstms, i figured there was no need to transform the input?

[hycis]

yap, if you do not wish to add the mlp layers before the lstm layers, then there's no need to add the transform. The bidirectionallstm takes three dimensional input.

[justiceamoh]

I see. so do you have any idea why it's throwing that error even without the transform? Any suggestions as to what to look at?

TypeError: ('Bad input argument to theano function with name "build/bdist.macosx-10.5-x86_64/egg/keras/models.py:127"  at index 1(0-based)', 
'Wrong number of dimensions: expected 3, got 2 with shape (20, 1).')

[hycis]

Oh because the output from the bidirectional is 3d, so you have to reshape the output before feeding to the mlp

model.add(BiDirectionLSTM(128,128, output_mode='sum’)) model.add(Reshape(128 * MAX_LEN)) model.add(Dense(128 * MAX_LEN,1,activation='sigmoid'))

On 11 Jul, 2015, at 2:18 am, Justice Amoh notifications@github.com wrote:

I see. so do you have any idea why it's throwing that error even without the transform? Any suggestions as to what to look at?

TypeError: ('Bad input argument to theano function with name "build/bdist.macosx-10.5-x86_64/egg/keras/models.py:127" at index 1(0-based)', 'Wrong number of dimensions: expected 3, got 2 with shape (20, 1).') — Reply to this email directly or view it on GitHub.

[justiceamoh]

oh right, that makes sense. thanks so much!