Usage question: constrained optimization satisfy just one constraint out of several

[michaelyli]

I'd like to run a constrained optimization such that a point is within the constrained set as long as it satisfies one of the constraints I impose. The default setting for constrained optimization requires a valid candidate point to satisfy all of the constraints.

Currently, I'm identifying the best point from a constrained optimization with each of the constraints separately. I basically create separate Bayesian optimization models with one of the constraints I'm considering. I then select the point with the highest acquisition score out of the best points from each of the separate constrained optimization problems as the next point to evaluate.

Is there a better way of doing this? I would really appreciate any help.

[apaleyes]

The only alternative I see here is to implement your own optimizer, maybe inheriting from the one in the package, that would provide this kind of customized behavior.

[michaelyli]

As a followup, do you have any advice for overriding certain functions within the GPyOpt library? I'm realizing that I could effectively solve my problems if I modified the acquisition_function function in this file https://github.com/SheffieldML/GPyOpt/blob/master/GPyOpt/acquisitions/base.py and the indicator_constraints function in this file https://github.com/SheffieldML/GPyOpt/blob/master/GPyOpt/core/task/space.py.

I do not know how to selectively override these functions and advice on how to proceed would be greatly appreciated.

[apaleyes]

One option would be to use Modular Bayes Opt class that allows you to define your problem in terms of objects. This way you can re-use some components of GPyOpt while providing your own implementations for others. Tutorial notebook where we override the acquisition function: https://github.com/SheffieldML/GPyOpt/blob/master/manual/GPyOpt_modular_bayesian_optimization.ipynb

[michaelyli]

I assume it's also possible to override the DesignSpace component as well?

[apaleyes]

Yes, although I would advise to exercise caution while doing that, given how central parameter space object is in the library

[michaelyli]

def indicator_constraints(self, x):
    x = np.atleast_2d(x)
    I_x = np.ones((x.shape[0],1))    

    if self.constraints is not None:
        for d in self.constraints:
            try:
                exec('constraint = lambda x:' + d['constraint'], globals())
                ind_x = (constraint(x)<0)*1
                I_x = I_x + ind_x.reshape(x.shape[0],1)
            except:
                print('Fail to compile the constraint: ' + str(d))
                raise

        I_x = (I_x >= 1).astype(float)
        return I_x

    else:
        return I_x

I've implemented the function above within my own custom Design_space class to override the default indicator constraints function. Basically, I'm allowing a point to be feasible as long as it satisfies one of the constraints (an "or" instead of the default "and"). I'm then passing this as a class to the modular Bayesian optimization class along with default classes for the other required arguments (as demonstrated in the Modular Bayesian Optimization tutorial).

I've tested a few different constraints. However for this particular set of constraints, when I call run_optimization, the call doesn't terminate.

constraints = [{'name': 'constr_1', 'constraint': '-x[:,1] + 0.5'}, {'name': 'constr_1', 'constraint': 'x[:,1] + 0.5'} ] I'm wondering if anyone has an idea of what may be causing the non-terminating behavior. Any advice would be greatly appreciated!

[michaelyli]

Here is the link to the full code. OverrideSpace.txt

[sdrobert]

@youngapsmikes @apaleyes The GPyOpt code re-initializes the design space during the optimization here. Also, as I've pointed out in #94, the anchor point generator calls constraints on both the expanded and non-expanded variables, so be careful.