Offer iterator interface

[GiggleLiu]

Using iterator interface, we can make the optimizer like a white box,

for info in optimize_iterator(func, x0, solver)
    println("Step = ", info["istep"])
    println("x = ", info["x"])
    println("f = ", info["f"])
   solver.PARAMETER = PARAMETER
    ...
end

This kind of interface is very welcome is debugging, tunning optimizer et. al. Please consider ~

A python version is of white box optimizer is https://github.com/BRML/climin

[pkofod]

Well, this is actually already sort of possible, but we just havn't explained how to do it anywhere. Look at the main loop for most of the algorithms here https://github.com/JuliaNLSolvers/Optim.jl/blob/master/src/multivariate/optimize/optimize.jl#L22 . We can make an explicit version like the on you show. Each next call would just iterate on the loop shown in the code I linked to.

[GiggleLiu]

@pkofod Cool, as it is a simple interface trick, will you do it immediately or need a pr?

[pkofod]

will you do it immediately or need a pr?

I don't have time right now, but if you want to have a go at a PR, I'd be happy to review and help. Otherwise, I'll get to it as soon as I can.