Closed hfxunlp closed 7 years ago
I am not good at both coding and english, I appreciate any kinds of your help.
👍 I think this module does not need to be restricted to SeqLSTM, SeqGRU etc. Also it would be helpful if this module could support :remember('both')
.
For anyone interested why you want to do this, read: http://www.stat.berkeley.edu/~tsmoon/files/Conference/asru2015.pdf (edit: sorry for some reason I posted the wrong link here)
@JoostvDoorn Thank you for your support. This module was considered to used before SeqRNN implementation, and I'm not clear how to implement remember
in this module, I just used it like that:
nn.Sequential():add(nn.SeqDropout()):add(nn.SeqGRU(128,128))
In my understanding, remember was a method of AbstractSequencer
and maybe there should be some changes in SeqLSTM, SeqGRU etc, but sorry for I do not know how to make it work, the codes were to hard for me.
That's okay, someone else (or me) should be able to implement remember for this fairly easily. Regarding it being only for SeqGRU etc., you can use it like this nn.Sequential():add(nn.SeqDropout(0.2)):add(nn.Sequencer(nn.GRU(128,128)))
. So we do not need to restrict this in the documentation.
@JoostvDoorn Thank you for your help, I do not have enough experience in this area, your advise is very helpful for me.
I change the Dropout implentation of Torch/nn, get a dropout module that could be used for SeqLSTM, SeqGRU and SeqBRNN. the whole sequence will share the same mask which is diferrent from nn.Dropout.