search

patrick-kidger / optimistix

Nonlinear optimisation (root-finding, least squares, ...) in JAX+Equinox. https://docs.kidger.site/optimistix/
Apache License 2.0
333 stars 14 forks source link

Least squares performance #88

Open Joshuaalbert opened 3 weeks ago

Joshuaalbert commented 3 weeks ago

Hi Patrick, I see you are using general solvers to solve the least squares problem, by contructing an objective out of residuals. I have found this approach is less performant than using Levenberg Marquardt. Do you get good performance?

patrick-kidger commented 3 weeks ago

Levenberg-Marquardt is already one of the possible solvers we have available.

What you are observing is that, in addition to that, we also support converting the residuals into a minimization problem if desired.

Joshuaalbert commented 3 weeks ago

Ah yes, as I explored more I soon saw you have LM. It would be interesting to add a benchmark comparing LM to minimiser. I can suggest a problem where one sees the difference.