Providing New Hyperparameters

salesforce / awd-lstm-lm

LSTM and QRNN Language Model Toolkit for PyTorch

BSD 3-Clause "New" or "Revised" License

1.96k stars 488 forks source link

Providing New Hyperparameters #9

Closed keskarnitish closed 7 years ago

keskarnitish commented 7 years ago

Some of the hyperparameters needed to be changed because of the PyTorch 0.2 port and discovery of the dropout bug. The new hyperparameters should reproduce (close to, or even better) the perplexity numbers on PTB and WT2 from the paper.

Smerity commented 7 years ago

LGTM. The Salesforce CLA bot was due to an earlier commit which didn't tie to @keskarnitish and has been fixed. Glad to get the hyper params out there for others :)