Potential diverging of SGD

airoldilab / sgd

An R package for large scale estimation with stochastic gradient descent

62 stars 18 forks source link

Potential diverging of SGD #94

Open ptoulis opened 7 years ago

ptoulis commented 7 years ago

We have seem this behavior when using constant learning rates with logistic regression models. In theory the iterates should not diverge with constant rates but bounce around the true value. This behavior is still not confirmed but would be helpful if anyone could run an independent experiment.