Example newton solver using MathProgBase

[mlubin]

Tidied up from my ISMP talk.

CC @dpo @tkelman for review

[tkelman]

Isn't github supposed to be able to render these now? Does it need to replace spaces in the filename with underscores or something?

nevermind, figured it out

[mlubin]

You can get a preview here: https://github.com/JuliaOpt/juliaopt-notebooks/blob/newton/notebooks/MathProgBase%20Newton%20solver.ipynb

[yeesian]

Will be nice if github renders nbdiff someday

[dpo]

Very nice. Thank you for posting this. This notebook is a great starting point for more elaborate solvers. A few minor comments:

• Maybe make the objective slightly more complicated than quadratic just so Newton's method takes more than one iteration?
• I find it odd that the Newton type actually says nothing about Newton's method, but is only related to the fact that you're using quadratic models with exact second derivatives. Maybe it should have a more evocative name.
• Is the name MathProgBase.optimize!() common to all optimization methods? I can't guess what method will be used from the name. I'm still learning JuMP and MathProgBase, but I'd rather write something like newton(model) rather than optimize(newton) where, in this last case, newton isn't an object related to Newton's method. I suppose I could define

C2Model = Newton
newton_solve(model :: C2Model) = MathProgBase.optimize!(model)

to make it clearer. It depends on whether the user is interested in knowing what's going on or not.

[tkelman]

Looks good. Certainly a nice toy demo which would've been good to have when first hooking things up to MPB, as opposed to having to reverse-engineer what JuMP and existing solver bindings are doing.

[mlubin]

@dpo, thanks for you comments, sorry it took so long to respond.

Maybe make the objective slightly more complicated than quadratic just so Newton's method takes more than one iteration?

I thought about this originally, but I'll leave this an an exercise to the reader. I'd prefer to keep the model as simple as possible for clarity; the goal isn't to test out newton's method but rather to show how to put all the pieces together.

I find it odd that the Newton type actually says nothing about Newton's method, but is only related to the fact that you're using quadratic models with exact second derivatives. Maybe it should have a more evocative name.

I renamed Newton to NewtonData and added a short comment on the purpose of the type: "The NewtonData type is a subtype of the AbstractMathProgModel type and stores all the instance data we need to run the algorithm."

Is the name MathProgBase.optimize!() common to all optimization methods? I can't guess what method will be used from the name.

This is Julia's multiple dispatch here. The optimize!() method dispatches on the specific type of the AbstractMathProgModel. Any parameters for the algorithm (or meta-parameters to choose the algorithm) are passed to the MathProgBase.AbstractMathProgSolver object earlier in the code, not at the call to optimize!. This isn't really meant to be the ultimate user-facing API (except for advanced users).