issues
search
accosmin
/
nano
C++ library [machine learning & numerical optimization] - superseeded by libnano
MIT License
1
stars
0
forks
source link
Variance-scaled stochastic gradient descent
#177
Closed
accosmin
closed
7 years ago
accosmin
commented
7 years ago
crazy idea: scale (minibatch-averaged) gradient with its variance: dx = avg(g) / (1 + var(g))
use a simple decreasing learning rate schedule