search

Element-Research / rnn

Recurrent Neural Network library for Torch7's nn
BSD 3-Clause "New" or "Revised" License
941 stars 313 forks source link

Implementation Request: dropout between gates in LSTM #335

Closed supakjk closed 7 years ago

supakjk commented 8 years ago

http://arxiv.org/abs/1512.05287 shows the usage of dropout between gates in LSTM. It would be great if we can also easily use such "inside" dropouts in rnn modules like FastLSTM. Keras already supports it https://keras.io/layers/recurrent/

jnhwkim commented 8 years ago

@supakjk bayesian dropout on GRU is implemented but yet to LSTM. However, I've already implemented it recently for my experiment. When the test is done, I'll send a PR including testcases.

supakjk commented 8 years ago

It would also be great if we can easily set the initial forget gate bias.

jnhwkim commented 8 years ago

Can you check the implementation? You can use nn.LSTM or nn .FastLSTM .

You can access submodule via self.recurrentModule.

jnhwkim commented 7 years ago

Refer to #382.