search

keitakurita / Better_LSTM_PyTorch

An LSTM in PyTorch with best practices (weight dropout, forget bias, etc.) built-in. Fully compatible with PyTorch LSTM.
MIT License
133 stars 20 forks source link

Variational Dropout Implementation #2

Closed anlijuncn closed 5 years ago

anlijuncn commented 5 years ago

hi, thanks for your sharing. I found that in your model.py, you constructed self.indrop and self.outdrop once, would it cause the dropout masks unchanged across batchs not only across time steps?

keitakurita commented 5 years ago

The indrop and outdrop layers each compute a different mask for each forward pass s othe mask will change across batches