search

karpathy / char-rnn

Multi-layer Recurrent Neural Networks (LSTM, GRU, RNN) for character-level language models in Torch
11.59k stars 2.58k forks source link

question about splitTable in model/LSTM.lua #175

Open YitzhakSp opened 8 years ago

YitzhakSp commented 8 years ago

in model/LSTM.lua we have the following code sequence

local reshaped = nn.Reshape(4, rnn_size)(all_input_sums) local n1, n2, n3, n4 = nn.SplitTable(2)(reshaped):split(4)

my question is about the second line. if I understand correctly here the (4, rnn_size)-tensor is being split in 4 tensors. now SplitTable(2) means that the split goes along the second dimension. wouldn't it be more natural to split along the first dimension or am I missing something ?

bearpaw commented 8 years ago

@YitzhakSp I think the first dimension corresponds to the "batchsize".

SJTUsuperxu commented 7 years ago

@YitzhakSp @bearpaw yes, the first dim should correspond to the batch_size. Maybe when we call the lstm.lua in train, the model has been utilized automatically for parallel processing, which I mean the input size comes to batch_size * vocab_size