search

google / jaxopt

Hardware accelerated, batchable and differentiable optimizers in JAX.
https://jaxopt.github.io
Apache License 2.0
909 stars 64 forks source link

Question about constrained problem solving. #395

Open DoTulip opened 1 year ago

DoTulip commented 1 year ago

Hello, everyone! I am troubled by a problem and want to ask for your help. I'm doing some research with neural networks, and the objects I'm studying have some constraints (both equality and inequality constraints). Optimizers like Adam only work for unconstrained problems, what can I do to elegantly handle these constraints (I tried the penalty function method, but it doesn't seem to work very well)?

mblondel commented 1 year ago

Unfortunately, we don't have support for constrained stochastic solvers at the moment. Our constrained solvers are for the full batch setting. If your constraints are not too complicated, maybe there is a change of variable to rewrite the problem in unconstrained form? For instance, if the constraints are {w : w >= 0, sum(w) = 1}, you can use the change of variable w = softmax(v) and minimize w.r.t. v instead.

DoTulip commented 1 year ago

@mblondel Thank you for your reply! It helps me a lot!