I'm new to Lua/Torch, trying to understand the initial lines of karpathy / char-nn LSTM which are:
local inputs = {}
table.insert(inputs, nn.Identity()()) -- x
for L = 1,n do
table.insert(inputs, nn.Identity()()) -- prev_c[L]
table.insert(inputs, nn.Identity()()) -- prev_h[L]
end
It's blowing out a few lines further on, because inputs is an empty list. It seems that nn.Identity()() always returns nil, and table.insert(randomTable, nil) does nothing. n is 2
[edited]
It appears that this code was written on the assumption that nn.Identity()() would return the Tensor object that is the output of the Identity node, and that this object would not change (i.e. for a given Identity node x,the value of x() at any time would be == x() at any later time. However, there is a kerfuffle in the history for nn.Identity about exactly this point, with a cycle of edits and reversions. If I read the history right, since Nov 27 2015, x() actually resets self.output to nil before returning it.
Now, the nn-char LSTM code was committed shortly before that, Sep 20 2015. But is it likely that Karpathy's char-nn has been broken almost since it was originally committed? Or am I looking at a wrong old version of torch somehow or what?
I'm new to Lua/Torch, trying to understand the initial lines of karpathy / char-nn LSTM which are:
local inputs = {} table.insert(inputs, nn.Identity()()) -- x for L = 1,n do table.insert(inputs, nn.Identity()()) -- prev_c[L] table.insert(inputs, nn.Identity()()) -- prev_h[L] end
It's blowing out a few lines further on, because inputs is an empty list. It seems that nn.Identity()() always returns nil, and table.insert(randomTable, nil) does nothing. n is 2
[edited]
It appears that this code was written on the assumption that nn.Identity()() would return the Tensor object that is the output of the Identity node, and that this object would not change (i.e. for a given Identity node x,the value of x() at any time would be == x() at any later time. However, there is a kerfuffle in the history for nn.Identity about exactly this point, with a cycle of edits and reversions. If I read the history right, since Nov 27 2015, x() actually resets self.output to nil before returning it.
Now, the nn-char LSTM code was committed shortly before that, Sep 20 2015. But is it likely that Karpathy's char-nn has been broken almost since it was originally committed? Or am I looking at a wrong old version of torch somehow or what?
I'm looking at: https://github.com/karpathy/char-rnn/blob/master/train.lua https://github.com/karpathy/char-rnn/blob/master/model/LSTM.lua https://github.com/torch/nn/blob/master/Identity.lua https://github.com/torch/nn/blob/master/Module.lua