Need some explanation on recurrent.py

[hugman]

First of all, thank you for great code. From your code, I got some good understandings to extend Keras API to my purpose.

I think your documentation link is broken. (or under construction?) ( https://seya.readthedocs.org )

Could you give some brief explanations on each class in seay.layers.recurrent.py ? Unfortunately, there are almost no comments in your code :)

Bidirectional GRUM ExoGRUM GRUM2 SingleCell

What I'm planning to do is implementing gru_cond_layer - Conditional GRU RNN in https://github.com/kyunghyuncho/dl4mt-material/blob/master/session2/nmt.py which is the theano code of http://arxiv.org/pdf/1409.0473.pdf

I think I can get good starting points from you code.

[EderSantana]

For now, forget about everything but Bidirectional. Everybody else is WIP and may just be deleted.

You can use it defining two RNNs and passing them as input:

gru1 = GRU(inp, out)
gru2 = GRU(inp, out)
brnn = Bidirectional(gru1, gru2)

model = Sequential()
model.add(brnn)
model.add ...
...

I hope that helps.

[hugman]

Ok. I see. Thank you.

[EderSantana]

I'll go ahead and close this. Reopen if you have more questions on how to use Bidirectional.

[daquang]

Have you tested Bidirectional? I get errors when I try using it on the imdb example data.

For example, it complains about return_sequences not being defined.

[EderSantana]

@daquang tbh, I've tested with an older Keras API and haven't write tests. But thanks for the report I'll check it out right now.

[EderSantana]

done! check this out https://github.com/EderSantana/seya/blob/master/examples/imdb_brnn.py

[daquang]

Still getting errors. What version of theano and keras are you using?

[EderSantana]

keras is the last one on github master, Theano is this:

git+https://github.com/Theano/Theano.git@15c90dd3#egg=Theano==0.8.git

Did you update seya? What is the error message?

[daquang]

I'm not using the latest github versions, just the latest stable versions. Theano 0.7 and Keras 0.1.3. Here is my error:

Traceback (most recent call last): File "imdb_brnn.py", line 67, in model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=4, validation_data=(X_test, y_test), show_accuracy=True) File "/home/dxquang/anaconda/lib/python2.7/site-packages/keras/models.py", line 490, in fit shuffle=shuffle, metrics=metrics) File "/home/dxquang/anaconda/lib/python2.7/site-packages/keras/models.py", line 211, in _fit outs = f(*ins_batch) File "/home/dxquang/anaconda/lib/python2.7/site-packages/theano/compile/function_module.py", line 606, in call storage_map=self.fn.storage_map) File "/home/dxquang/anaconda/lib/python2.7/site-packages/theano/compile/function_module.py", line 595, in call outputs = self.fn() ValueError: dimension mismatch in args to gemm (32,64)x(128,1)->(32,1) Apply node that caused the error: GpuDot22(GpuJoin.0, <CudaNdarrayType(float32, matrix)>) Inputs types: [CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, matrix)] Inputs shapes: [(32, 64), (128, 1)] Inputs strides: [(64, 1), (1, 0)] Inputs values: ['not shown', 'not shown']

[EderSantana]

I my experience old theano had problems with concatenation, which is what seems to be happening here. The theano that comes with anaconda is not good. Give the version I just told you a try. You can always do that in a virtualenv

pip install git+https://github.com/Theano/Theano.git@15c90dd3#egg=Theano==0.8.git

[yinruiqing]

@EderSantana I want use Layer brnn as the first layer. But I met this error. The first layer in a Sequential model must get aninput_shapeorbatch_input_shapeargument. I've set the input_shape in forward lstm and backward lstm. Can you tell me how to solve this problems?