In optim.sgd, weight decay is not subject to learning rate decay

[rolfe]

Weight decay uses the undecayed learning rate. As a result, as training progresses, the weight decay is effectively given more and more emphasis. This does not seem correct.

[clementfarabet]

Good point. Do you want me to change that? It makes sense to me.

[clementfarabet]

It was actually still incorrect. I've just submitted a new change, it should be correct now.

For luarocks users, I've created a new version, so that older experiments can still be reproduced.