Enable warm-starting the hessian approximation in L-BFGS

[zaccharieramzi]

Currently one can only provide an initial estimate of the solution, enable warm start of the iterates. But for quasi-Newton methods, it can also be a good idea to provide initial estimates of the hessian approximation, typically when solving multiple time a similar problem.

This was for example done in HOAG by @fabianp (see https://github.com/fabianp/hoag/blob/master/hoag/hoag.py#L109).

I am willing to implement this in the next few weeks.

As I know it is of interest to them as well, cc-ing @marius311 and @mblondel

[mblondel]

That would be useful but we need to think of an API to do it. Maybe with a named tuple:

init = LBFGSInit(params=params, grad=grad, hessian=hessian)
res = lbfgs.run(init, *args, **kwargs)

[fabianp]

indeed, I think this would be very useful!