Keras based gridLSTM

[ypxie]

It's great to find this implementation, I have been looking for this for long.

I am not sure does the keras based GridLSTM work ? or you already shifted to torch7?

[davidbuniat]

No unfortunately Keras based implementations is not finished experimentation. I had some difficulties with Scan() though I guess one could overcome them. (The problem is storing weights in multidimensional iterations)

GridLSTM works on torch (yet it is not validated but I am working on it). One could easily transform torch implentation into Keras. In case you need help in understanding how everything works let me know. Otherwise, always good to learn Torch, even though for RNNs, the only limitation is that you must do unfolding that takes lots of memory space and limits the model for images.

[ypxie]

It seems that you are using 1d data to imitate 2d data to skip the nested loop in theano.

For two dimensional grid, padding are applied on both row and column. if you use [-1, -Row] to represent the row and column predecessors, how to represent the column padding?

I am not sure if I explain my question clearly.

[davidbuniat]

I have gone through your code its a little bit confusing for me as now I am concentrated into Torch. (Better not to change my abstraction to Theano again)

What would I suggest to do from my experience of past month,

1) is to develop MVP with for-loops without any scan() or complicated/limited tool. Do not care about the speed, apparently haven't seen any MD RNN based on GPU. Please refer to this paper. They do everything on CPU and achieve good results.

2) Secondly setup experiment environment exactly the same as it is done in GridLSTM paper or Multidimensional RNN paper and try to achieve the same results with same configuration. Only then you will be sure that you have developed right model.

3) Do Optimization e.g. try to use Scan() and run the experiment to see that nothing breaks.

4) Do your own experiment with achieved model.

Hope this will help. :) Good luck. Write me in case you are at least on finished 2 step.

[ypxie]

Thank you for your advices. I implemented several theano based 2D RNN, but the results are not promising.

I saw you also did some experiments on pybrain.
Does the multi-dim lstm work on pybrain? I run the pybrain_mdrnn.py but it gives me errors because of the dataset problem, could you upload the data set for pybrain? Thank you! .

[davidbuniat]

I can upload but those experiments with py_brain could not simply work as if I am not wrong at that time I couldn't find out any multidimensional container (object) to pass into the network.

But if you still want to give a try I will upload them.

[ypxie]

Thank you very much!

I also tried your Torch 7 implementation of GridLSTM, it works for cpu, but for GPU it rises several errors in the optim part. Did you also encounter this?

[davidbuniat]

here is the dataset I was working. datasets.zip

Haven't yet sorted out with GPU, but will do in upcoming weeks.

[ypxie]

Thank you very much!

[arita37]

Hello,

Just wondering if we have some details on grid lstm.

Thanks, regards.