Dropout with Expectation-linear Regularization

clab / dynet

DyNet: The Dynamic Neural Network Toolkit

Apache License 2.0

3.43k stars 704 forks source link

Dropout with Expectation-linear Regularization #989

Open danielhers opened 7 years ago

danielhers commented 7 years ago

Ma et al. (2017) introduce dropout with regularization (DR), the benefit of which in NLP is shown by Strubell et al. (2017). Does DyNet dropout support this regularization?

neubig commented 7 years ago

I doubt this is implemented, but if it is not easy to implement with the operations already provided by DyNet it might be nice to have.