Possibly Incorrect Learning Rate

thaihungle / SAM

Self-attentive Associative Memory & SAM-based Two-Memory Model

MIT License

46 stars 8 forks source link

Possibly Incorrect Learning Rate #3

Closed mind-matrix closed 3 years ago

mind-matrix commented 3 years ago

Hey. So I was going through the bAbI model config and the learning rate is fixed at 0.0006 whereas the paper suggests using a learning rate of 0.006 which is 10x the config learning rate. Is this a typo or am I missing something?

thaihungle commented 3 years ago

Sorry, this is a typo in the paper. The correct one is in the code.

mind-matrix commented 3 years ago

Thanks.