Closed takafusui closed 4 years ago
I think this is a reasonable use of the fail
flag. If line search reaches the maximum number of iterations, it means that we were unable to find a point along the search direction that improved the function value.
Given a deterministic function, this generally implies that we should stop optimizing given the settings that L-BFGS was run with, because if we ran another step we would encounter the same outcome.
In the document of Exact GP Regression with Multiple GPUs and Kernel Partitioning, you coded:
but according to the source code of PyTorch-LBFGS (line 486):
You might treat the fail flag in a different way, or maybe am I wrong? Thank you.
The example is found in: https://gpytorch.readthedocs.io/en/latest/examples/02_Scalable_Exact_GPs/Simple_MultiGPU_GP_Regression.html