Closed SundeepK closed 8 years ago
I was wondering if there is a missing sqrt call on this line https://github.com/karpathy/convnetjs/blob/master/src/convnet_trainers.js#L117
According to this article (links to adadelta paper)http://sebastianruder.com/optimizing-gradient-descent/index.html#adadelta it would suggest that:
var dx = - Math.sqrt((xsumi[j] + this.eps)/(gsumi[j] + this.eps)) * gij;
should be:
var dx = - (Math.sqrt(xsumi[j] + this.eps)/Math.sqrt(gsumi[j] + this.eps)) * gij;
Would be great if someone could correct my understanding if not.
Ah looks like I miss read the parenthesis! Looks fine.
I was wondering if there is a missing sqrt call on this line https://github.com/karpathy/convnetjs/blob/master/src/convnet_trainers.js#L117
According to this article (links to adadelta paper)http://sebastianruder.com/optimizing-gradient-descent/index.html#adadelta it would suggest that:
should be:
Would be great if someone could correct my understanding if not.