stevengj
commented
6 years ago I would generally recommend BOBYQA instead.
It's not really clear to me what to do when you hit a NaN value. It would be better to figure out why the NaN appears and catch the problem earlier, when you might have enough information to do something about it.
Sometimes the MMA algorithm gets stuck in an infinite loop when called by NEWUOA_BOUND. The problem seems to be in mma_minimize, when dd.gval is NAN.
I have suggestions for two potential fixes.
Fix 1: Add an if check at mma/mma.c, line 289, after dual_func() and before fcur=f(). Is it always true that we want to stop or are there cases where we would want to continue? if (nlopt_isnan(dd.gval)) { ret = NLOPT_FORCED_STOP; goto done; }
With this change, NEWUOA_BOUND propagates NLOPT_FORCED_STOP to the caller. Perhaps that is the right thing for NEWUOA to do. In my test problem it seems to be stopping at a local minimum, but I expect that is because the MMA sub-problem occurs at those minima, and it might stop away from a local minima for a different test function. My test function also caused NLOPT_ROUNDOFF_LIMITED to be returned, which leaves the caller uncertain if the best observed values are for a local minimum.
Fix 2: Implement NEWUOA_BOUND by wrapping the objective function, clipping the bounds and applying a quadratic penalty function. (Remove MMA code.) That way the quadratic fit could be well behaved at the boundary and the caller's function would not be evaluated at the bounds.
Callers can do that already, and we should add an example to the docs for how to do it. I tried a simple penalty, and although I don't know how robust it is for general purpose use, I observed a reduction in function evaluations and no roundoff or other errors from NEWUOA.
Here is function I observed the problem with, having bounds [-1, +1] for both (2) dimensions. http://infinity77.net/global_optimization/test_functions_nd_C.html#go_benchmark.CosineMixture def func_to_minimize(x): """CosineMixture test function""" return -0.1np.sum(np.cos(5.0np.pi*x)) - np.sum(x**2.0)