I'm trying to implement an LSTM extension that outputs a sentinel vector alongside the cell state and hidden. I did not want to mess with the existing LSTM.lua structure i.e. forward and backward calls, so I decided to re-implement it using Recurrence (for just a single time-step input). I ran into problems when my recurrent module has two outputs. Consider a simple example :
My rn module expects an input {inp,h[t-1]} == {inp,{cell,hidden}} but I cannot do rn:forward(torch.Tensor{12}) since the rm output is a table of length 2. I believe table output is not supported in Recurrence.lua. Here for the first time step, internally output is set to zero vector of 10x1.
For my project, I was considering modifying the nngraph of FastLSTM and passing that to the recurrence module, but this throws a spanner in my plan.
Any ideas on how to proceed? Or any re-implementation of LSTM using Recurence and Sequencer? I couldn't find any online
Hi,
I'm trying to implement an LSTM extension that outputs a sentinel vector alongside the cell state and hidden. I did not want to mess with the existing LSTM.lua structure i.e. forward and backward calls, so I decided to re-implement it using Recurrence (for just a single time-step input). I ran into problems when my recurrent module has two outputs. Consider a simple example :
My
rn
module expects an input{inp,h[t-1]} == {inp,{cell,hidden}}
but I cannot dorn:forward(torch.Tensor{12})
since therm
output is a table of length 2. I believe table output is not supported in Recurrence.lua. Here for the first time step, internally output is set to zero vector of 10x1.For my project, I was considering modifying the nngraph of FastLSTM and passing that to the recurrence module, but this throws a spanner in my plan. Any ideas on how to proceed? Or any re-implementation of LSTM using Recurence and Sequencer? I couldn't find any online