Element-Research / rnn

Recurrent Neural Network library for Torch7's nn
BSD 3-Clause "New" or "Revised" License
939 stars 313 forks source link

Can anybody write test case for this Bayesian LSTM? #382

Closed jnhwkim closed 7 years ago

jnhwkim commented 7 years ago

though this code is used in my recent work.

I implement Bayesian LSTM and can be applied TrimZero, which resolves #335.

nicholas-leonard commented 7 years ago

Don't have to write it. Current test past, so its fine for now.

hashbangCoder commented 7 years ago

Hi,

I have been testing the dropout, specifically for FastLSTM and there seems to be some implementation errors.

 p= 0.5    --dropout prob
lstm = nn.LSTM(10,20,nil,nil,p)
print(lstm.p)   --returns 0.5 and works as expected

-- But,
f_lstm = nn.FastLSTM(10,20,nil,nil,nil,nil,p)
print(f_lstm.p)   -- prints 0 !

I think the error is in calling the __init() of parent class in FastLSTM here. This overwrites the self.p of child class. If you comment that line for example (or preferably pass the p argument to __init), it works fine.

Just to be sure, I updated the module before checking. And I checked multiple times for argument positional errors. I hope I'm not wrong on the OOP part.

@nicholas-leonard @jnhwkim

jnhwkim commented 7 years ago

@hashbangCoder I'll check that shortly. Thanks!