Add RNN / LSTM examples for Keras in ModelZoo

[dexter1691]

Keras supports recurrent layers like LSTM / RNN / GRU etc. Right now model zoo for Fabrik doesn't contain any examples of network that uses these layers.

We should add a few examples to demonstrate the capability of using recurrent layers in a network and to test how import / export works for these layers.

[Ram81]

@dexter1691 can I take this?

[dexter1691]

@Ram81 : Go for it! Thanks!

[prakharchoudhary]

@dexter1691 Since there are multiple models from several domains that can be added, would it be okay for me to work on this issue as well?

[dexter1691]

@prakharchoudhary Ofcourse! Feel free to work on it. I think to ensure efforts are not duplicated, before working on adding the examples, mention the model here so that everyone is aware of it.

[Ram81]

@dexter1691 can you point us to some of the models!

[utsavgarg]

@Ram81 you could look at Captioning models (Show and Tell, Neuraltalk), VQA models, and Text generation models.

[Ram81]

@utsavgarg I am referring this one https://github.com/jeffdonahue/caffe/blob/recurrent-rebase-cleanup/examples/coco_caption/lstm_language_model.prototxt as of now but I would need keras models right?

[Ram81]

@utsavgarg would it be fine if I export this model https://github.com/fchollet/keras/blob/master/examples/lstm_text_generation.py to json then use it as an example?

[utsavgarg]

@Ram81 you could do it in Caffe or Keras, but choose one out of these two as they are quite similar.

[Ram81]

@utsavgarg I used this model https://github.com/jeffdonahue/caffe/blob/recurrent-rebase-cleanup/examples/coco_caption/lstm_language_model.prototxt & tested it in here's the model

But when I tried to export it to keras it says Can't determine size of HD5Data layer

[Ram81]

Also there are some examples available but I would need to obtain a json file through its implementation if you want I can test them too.

[utsavgarg]

@Ram81 When you are adding this network, can you also see how to handle the visualization for it better? Also, the error for export to Keras is expected so that is fine.

[Ram81]

@utsavgarg Okay I will take care of the visualization.

[dexter1691]

I think we should also try just importing Keras layers with recurrent layers too. That might be easier / more reliable since Keras's support for recurrent layer is much better than caffe. For instance, you can try importing model definition from here: https://github.com/fchollet/deep-learning-models/blob/master/music_tagger_crnn.py https://machinelearningmastery.com/text-generation-lstm-recurrent-neural-networks-python-keras/ https://github.com/mineshmathew/char_rnn_karpathy_keras/blob/master/char_rnn_of_karpathy_keras.ipynb https://github.com/afgiel/lstm_lm/blob/master/generate.py https://github.com/philipperemy/keras-attention-mechanism/blob/master/attention_lstm.py

[Ram81]

@dexter1691 I would need to export that keras model to a json file, right?

[utsavgarg]

@ram81 yes

[Ram81]

@utsavgarg I will create a PR for this model & start with adding examples for keras too.

[adbugger]

@dexter1691 , is it ok if I work on it too? Correct me if I'm wrong, but more examples will do no harm.

[utsavgarg]

Multiple examples added in #125, #134, #140 and #166.