CG_DESCENT linesearch (to improve)

accosmin / nano

C++ library [machine learning & numerical optimization] - superseeded by libnano

MIT License

1 stars 0 forks source link

CG_DESCENT linesearch (to improve) #41

Closed accosmin closed 5 years ago

accosmin commented 8 years ago

CG_DESCENT line search is not working like advertised in the relevant paper:

it takes many more function value & gradient evaluations
its speed is much lower than of the competing CGD & LBFGS variants

accosmin commented 8 years ago

Should run the benchmark using test functions with higher number of dimensions. The authors have shown results for more than 50 dimensions.

accosmin commented 8 years ago

https://www.math.lsu.edu/~hozhang/papers/lcg.pdf http://link.springer.com/article/10.1007%2Fs10589-013-9558-3#/page-1 http://www.researchgate.net/publication/256936445_Globally_convergent_modified_Perrys_conjugate_gradient_method

accosmin commented 5 years ago

In the original paper they use c1=0.1 and c2=0.9 which is different than the defaults suggested by Nocedal&Wright (c1=1e-4, c2=0.9) for CGD methods.

accosmin commented 5 years ago

Currently it produces similar results with the More&Thuente version when using an improved cg-descent-based step initialization and a safeguard interpolation.