search

stephenbeckr / L-BFGS-B-C

L-BFGS-B, converted from Fortran to C, with Matlab wrapper
BSD 3-Clause "New" or "Revised" License
108 stars 56 forks source link

Possibility for hot restarts #13

Open fvanmaele opened 1 month ago

fvanmaele commented 1 month ago

L-BFGS-B keeps m vector pairs <s_I,y_I>, where the oldest pair is discarded at every iteration exceeding m. I was wondering on an approach where L-BFGS-B could resume from m given vector pairs, with the vectors update performed manually.

Specifically, the pairs s_{I+1}, y_{I+1} would be computed from some other descent method independent from L-BFGS-B, without requiring a restart of L-BFGS-B that generally lowers performance.

A concrete use-case is multilevel optimization (e.g. https://doi.org/10.1137/08071524X) where a new point is updated on level h with objective f_h, based on information from a level h+1 with objective f_{h+1} of some lower dimension.

Thanks for the C and MATLAB implementation, not requiring a separate Fortran compiler is very useful on macOS. 😃

stephenbeckr commented 1 month ago

That does seem like a pretty good idea, and it could be turned off by default (i.e., only activated if an advanced user passes in the warm-start vectors) so it shouldn't interfere with legacy usage.

However, I'm not really able to spend time adding features at the moment or in the near future. But if you wanted to write that, and then make a pull request, I'd be happy to accept merge it