Closed pmelchior closed 7 years ago
Because this function is called each time X1, X2... get updated, we should make it fast and/or cheap. For the latter part, I suggest that the NMF creates a class with an iteration counter and a call function, which only calculates the constants when they are expected to change: during early iterations.
Once the algorithm gets close to convergence, the changes will be slow, and it can simply report the value of previous iteration(s) modulo some slack.
This is a trade-off between GLMM iterations and cost per iteration.
Currently the computation of the Lipschitz constant of grad f in update_steps() assumes that objective function is of the form
||X_1 X_2... X_N - Y||^2
, but that is not necessarily true. We need to be able to set the step sizes in a flexible way, based on the current value of allX_j
.I suggest that we provide a callback function.