search

pints-team / pints

Probabilistic Inference on Noisy Time Series
http://pints.readthedocs.io
Other
224 stars 33 forks source link

1-D line search alogrithms #1160

Open alisterde opened 4 years ago

alisterde commented 4 years ago

As we discovered when looking at the L-BFGS optimiser (#1083 ) line search algorithms are not that straightforward and are an integral part of Quasi-Newton and conjugate gradient methods. Furthermore, a lot of algorithms for such methods are reliant on legacy code implemented in FORTRAN. It would be extremely beneficial to have a working ask/tell line search in pints as it would pave the way to include numerous Quasi-Newton and conjugate gradient methods, while maintaining pints's lightweight low dependency ethos.

Good initial literature on the area is listed below: https://www.math.lsu.edu/~hozhang/papers/cg_descent.pdf http://www.caam.rice.edu/~yzhang/caam554/pdf/cgsurvey.pdf https://dl.acm.org/doi/pdf/10.1145/1132973.1132979

This shows the 'Hager-Zhang' line search would be of particular interest to implement.

This is part of #684.

alisterde commented 4 years ago

A starting points for this could be just getting a working line search algorithm (maybe even start just as a for loop) in a Jupyter notebook for a simply parabola with parameters of the same order of magnitude, increase the complexity of the model, then implement an ask/tell line search in the pints framework.