search

google / jaxopt

Hardware accelerated, batchable and differentiable optimizers in JAX.
https://jaxopt.github.io
Apache License 2.0
936 stars 66 forks source link

Levenberg Marquardt with box constraints? #607

Open N9-Tails opened 3 months ago

N9-Tails commented 3 months ago

Hello. I am wondering if there is any workarounds in the library to support box constraints during the Levenberg Marquardt optimization. Any guidance would help tremendously. Thanks in advance!

mblondel commented 3 months ago

You could try a change of variable, min f(x) s.t. L <= x <= U <-> min f(min(max(x, L), U)), potentially replacing min and max with soft approximations.

Joshuaalbert commented 2 months ago

As a suggesting, you can use these constraints to/from unit interval. It's faster than using something like sigmoid. https://github.com/Joshuaalbert/jaxns/blob/main/jaxns/internals/constraint_bijections.py#L11