SciML / NonlinearSolve.jl

High-performance and differentiation-enabled nonlinear solvers (Newton methods), bracketed rootfinding (bisection, Falsi), with sparsity and Newton-Krylov support.
https://docs.sciml.ai/NonlinearSolve/stable/
MIT License
221 stars 39 forks source link

Inexact Newton Steps in Levenberg Marquardt #319

Open avik-pal opened 7 months ago

avik-pal commented 7 months ago

See the last part of http://ceres-solver.org/nnls_solving.html#levenberg-marquardt. This is basically already there and this is mostly a internal solver selection mechanism:

  1. Add a forcing function to LM
  2. Add a newton_step = Val(:exact) / Val(:inexact) option. Set it to nothing by default
  3. Default to exact for factorization, else inexact
  4. Inexact Newton is just setting the tolerance correctly in dolinsolve.
avik-pal commented 6 months ago

https://www.sciencedirect.com/science/article/pii/S0377042705007958

avik-pal commented 4 months ago

With the redesign this is very easy to do now, we need to add forcing terms to the newton-style descent algorithms and all algorithms will be able to use it