How to keep all evaluations?

[TianningGao]

I wanted the solver to keep records of all evaluations but it didn't. I tried the simple example in the online doc with logging. Here's the code:

import numpy as np import pybobyqa

def rosenbrock(x): return 100.0 * (x[1] - x[0] 2) 2 + (1.0 - x[0]) ** 2

x0 = np.array([-1.2, 1.0])

params = { 'logging.save_diagnostic_info':True, 'logging.save_xk':True, 'logging.save_poisedness':False }

soln = pybobyqa.solve( objfun = rosenbrock, x0 = x0, maxfun = 10, objfun_has_noise = False, do_logging=True, print_progress=True, user_params = params)

print( soln.diagnostic_info )

As you can see, the max evaluation is 10. What I had in the printed diagnostic_info is only 5 records of evaluation, while I was expecting at least 10, including the starting point.

Is there a way to record all evaluations? What could be the cause of this issue?

[lindonroberts]

Hi @TianningGao,

The diagnostic information only records one line per iteration of the main algorithm (and shows the best evaluation so far). Unfortunately, each iteration can have more than one evaluation, so the diagnostic info doesn't show every evaluation.

Py-BOBYQA does not have a way of storing the entire history, so my way of doing this is to wrap your objective function in a class which stores the history (and implements the __call__ function).

class ObjfunWrapper(object):
    def __init__(self, objfun):
        self._objfun = objfun
        self._xvals = []
        self._fvals = []

    def __call__(self, x):
        # Evaluate objective, but save everything too
        self._xvals.append(x.copy())
        f = self._objfun(x)
        self._fvals.append(f)
        return f

    def get_results(self):
        return np.stack(self._xvals), np.array(self._fvals)

# Example usage, based on your above code
my_objfun = ObjfunWrapper(rosenbrock)
soln = pybobyqa.solve(objfun=my_objfun, x0=x0, maxfun=10)
print(soln)
xvals, fvals = my_objfun.get_results()
print(xvals)  # NumPy matrix, each row is the vector x used for the evaluation
print(fvals)  # each entry is the corresponding f(x) for each evaluation

[TianningGao]

Thanks for your solution. I'm still confused why Py-BOBYQA evaluate one point multiple times even if I set "objfun_has_noise" to "False"?

[lindonroberts]

It shouldn't evaluate one point multiple times, but it sometimes evaluates more than one point per iteration (one evaluation is the next candidate iterate, but there may be other evaluations used to update the internal model for the objective).

[TianningGao]

OK, I see. Thank you so much.