Early termination issue in lbfgsb

[Xiaoqian-Liu]

Hello, I encountered an issue when calling the lbfgsb function. When I used the default settings, I always got a non-optimal solution. I wanted to run the algorithm longer to see if the algorithm could converge to a better solution. I set 'factr=0' and 'pgtol=0' to avoid early terminations and expected that the algorithm would run 'maxIts' iterations based on the value I set to 'maxIts' (if I set 'maxTotalIts' large enough). However, the algorithm always terminates after 5 iterations, no matter how large the values I set to 'maxIts' and 'maxTotalIts'. If I set very small values to 'maxIts', say 3, the algorithm terminated after 3 iterations. So, I believe the 'maxIts' parameter works. My guess is that there is another hidden early stopping rule for the lbfgsb function. Is that correct? Is there any way for me to modify the code to make the algorithm run longer? Thanks very much!

[stephenbeckr]

Sorry to hear that. Not sure if it's a wrapper issue or an issue in the underlying code, which I can't support. If you have a short MRE then I might be able to help.

[Xiaoqian-Liu]

Hello, Stephen. Thanks for the response. We can see the early stopping issue using the examples in your code. For example, I tested the Rosenbrock function and saw the same issue. Please see my code below. I would appreciate it a lot if you can provide some input.

` disp('=== lbfgsb "driver1" test problem (Rosenbrock, 10 dimensions) === '); % Here's the test problem included with lbfgsb called 'driver1' % (It's a version of the Rosenbrock test function)

n = 10;

l = ones(n,1); u = l; odd = 1:2:n; even= 2:2:n; l(odd) = 1.0; u(odd) = 1.0e2; l(even)= -1.0e2; u(even)= 1.0e2;

opts = struct( 'x0', 1e4*ones(n,1) ); opts.printEvery = 100; % controls how often we print output from .m file wrapper opts.m = 10; opts.maxIts = 3e3; opts.pgtol = 0; opts.factr = 0; opts.errFcn = @(x) norm(x-1); % for now just an arbitrary fcn % opts.verbose = -1; % default is -1, i.e., no output from mex

[x,f,info] = lbfgsb( @driver1, l, u, opts );

% The true objective value is 0. if abs(f) < 1e-8 disp('Success!'); else disp('Something didn''t work right :-( '); end `

Some modifications I made:

1. I set an initial value very far away from the optimal solution.
2. I set factr=0 and pgtol=0 to avoid the two early stopping rules. You can see that the algorithm stops at iter=2263 when I set maxIts=3e3, which was not what I expected. The final function value is around 1e-5 and the infinity norm of the gradient is around 1e-3, both of which are not close enough to the optimal values. Any thoughts on how to make the algorithm run longer? Thanks!

[stephenbeckr]

Hi Xiaoqian,

I see. You're right, there is another stopping condition. If you run help lbfgs you'll see the documentation I made which helps with this sort of thing. In this case, the issue is that maxTotalIts is at its default value of 5000. If you run the example code above, upon exist, info.totalIterations is 5000.

So the solution is to just increase opts.maxTotalIts to something much larger. I think that solves it, so I'm closing out this ticket. -Stephen