search

JuliaNLSolvers / Optim.jl

Optimization functions for Julia
Other
1.12k stars 217 forks source link

IPNewton / ConstrainedOptim TODOs #609

Open anriseth opened 6 years ago

anriseth commented 6 years ago

This was originally at https://github.com/JuliaNLSolvers/ConstrainedOptim.jl/issues/4

Here's a list of things that I believe we should focus on next. If anyone is interested in helping out here that would be very much appreciated.

Interface

Interior / IPNewton

Linesearch

Testing

Docs

kkmann commented 5 years ago

Hey, just wanted to bump this since I just recently used the constrained optimization functionality in Optim.jl - thanks a lot for implementing this! I wanted to use ForwardDiff and followed the IPNewton tutorial as close as possible. I.e. I implemented all gradients, Jacobians, and Hessians myself via ForwardDiff - is there a similar functionality like autodiff = :forward for non-constrained problems (if not, that would be cool ;))? Also, I had a hard time figuring out that the mu parameter is quite important - had to chose it manually to get a solution that respected the constraints.

anriseth commented 5 years ago

Happy to hear that you've used this:)

There is an automatic differentiation convenience call for first order, but not for second order. See https://github.com/JuliaNLSolvers/NLSolversBase.jl/blob/master/src/objective_types/constraints.jl#L122

If you want to extend it to TwiceDifferentialbleConstraints that would be very helpful; PRs welcome :)

When I looked at convenience functions for constraint differentiation I stopped at first order because (1) I wasn't sure at the time what the best way to evaluate the Hessian part of the constraints, and (2) my view is that most cases where we don't have explicit second-order functions it's better to just LBFGS.

Unfortunately I never got around to implement interior point (L)BFGS. I've started working in industry now and don't user Julia anymore, so I won't be fixing any of the to-dos myself.

anriseth commented 5 years ago

Regarding ~my~mu: I'm glad you figured out how to set it manually. Can you please add a section on fixing constraint violations in the IPNewton docs, here https://github.com/JuliaNLSolvers/Optim.jl/blob/master/docs/src/algo/ipnewton.md

If you don't know how, please let us know and we'll give further instructions :)

antoine-levitt commented 5 years ago

I've started working in industry now and don't user Julia anymore, so I won't be fixing any of the to-dos myself.

On behalf of both academia and the Julia ecosystem, sorry to hear that! Best of luck :-)

anriseth commented 5 years ago

On behalf of both academia and the Julia ecosystem, sorry to hear that! Best of luck :-)

Thanks for saying that, much appreciated! I'll probably follow the developments in the Julia-verse and hope I get to come back to it someday :)

kkmann commented 5 years ago

Okay, It's been a while since I fiddled with Documenter - your make.jl complains about a missing site name on my system. Are there any special twists and whistles to your Documenter setup?

Concerning the 2nd order autodiff, I'll have a look at it. Might not be an ideal solution but I got it working for my problem already :)

anriseth commented 5 years ago

I used to set up Documenter to use format = :html when testing locally. You can follow the setup at https://github.com/JuliaNLSolvers/LineSearches.jl/blob/master/docs/make.jl#L15 Just include the relevant docs page in the pages keyword.