search

pmelchior / proxmin

Proximal optimization in pure python
MIT License
109 stars 23 forks source link

Lipschitz constant needs to be configurable. #2

Closed pmelchior closed 7 years ago

pmelchior commented 7 years ago

Currently the computation of the Lipschitz constant of grad f in update_steps() assumes that objective function is of the form ||X_1 X_2... X_N - Y||^2, but that is not necessarily true. We need to be able to set the step sizes in a flexible way, based on the current value of all X_j.

I suggest that we provide a callback function.

pmelchior commented 7 years ago

Because this function is called each time X1, X2... get updated, we should make it fast and/or cheap. For the latter part, I suggest that the NMF creates a class with an iteration counter and a call function, which only calculates the constants when they are expected to change: during early iterations.

Once the algorithm gets close to convergence, the changes will be slow, and it can simply report the value of previous iteration(s) modulo some slack.

This is a trade-off between GLMM iterations and cost per iteration.