the learning rate of word_language_model

pytorch / examples

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.

https://pytorch.org/examples

BSD 3-Clause "New" or "Revised" License

22.29k stars 9.53k forks source link

the learning rate of word_language_model #757

Open zhangyingbit opened 4 years ago

zhangyingbit commented 4 years ago

Hi, I have a question about the learning rate in the example "word_language_model", the init lr = 20, which seems very large, can you tell me why lr is set to equal 20? Thanks a lot! If you have some advices about improving the performance, please let me know and thanks

kienld3049 commented 2 years ago

i also have the same question. Look like nobody answers that

AlbertMa123 commented 6 months ago

me too, when i used lr=20, then the loss was very strange and crazy......

msaroufim commented 6 months ago

I'm happy to accept contributions for a more reasonable learning rate