what is the advantage of using LinearNB over Linear with bias in LSTM for input-to-hidden unit maps?

Element-Research / rnn

Recurrent Neural Network library for Torch7's nn

BSD 3-Clause "New" or "Revised" License

939 stars 313 forks source link

what is the advantage of using LinearNB over Linear with bias in LSTM for input-to-hidden unit maps? #297

Closed eriche2016 closed 8 years ago

eriche2016 commented 8 years ago

can you give some explainations or some reference. thanks

eriche2016 commented 8 years ago

I kind of know the difference, is it try to avoid the 'double' biases term. for example, when summation of two linear layer outputs will cause two biases term here, this kinda violate the equation of RNN or LSTM. am i correct?

nicholas-leonard commented 8 years ago

@eriche2016 Yes, exactly.