A question about _drop_weights()

keitakurita / Better_LSTM_PyTorch

An LSTM in PyTorch with best practices (weight dropout, forget bias, etc.) built-in. Fully compatible with PyTorch LSTM.

MIT License

133 stars 20 forks source link

A question about _drop_weights() #7

Open Achronferry opened 4 years ago

Achronferry commented 4 years ago

I notice that the model drops _weighthh in forward() and never recover them. The dropped neural unit will keep to 0 and never be recovered. After a series of turns, Every element of the weight matrix will be 0

zhou13 commented 3 years ago

I also have the same question. The code looks wrong to me if dropoutw > 0.

konstantinosKokos commented 6 months ago

+1 Implementation is blatantly incorrect