claesenm / optunity

optimization routines for hyperparameter tuning
http://www.optunity.net
Other
416 stars 78 forks source link

Python: Inequality constraints and bounds for several parameters #82

Open ghost opened 7 years ago

ghost commented 7 years ago

Hi. I am interested in optimizing several parameters of a model that is defined as y = f(x), where x is a vector of all the unknown parameters that I am interested in finding. I have been able to use the methods from scipy.optimize.minimize to solve this problem with constraints, though I am not satisfied with the performance of the optimized values...so, I am trying to see if optunity can give better results.

I couldn't find any help on how to implement an inequality constraint that looks like following (i.e. not specific to a single parameter, but applied on multiple parameters together): 1 - sum(x_guess) > 0

In the above constraint, x_guess is a single argument to my objective function y, although x_guess is a vector containing values that represent several parameters I am interested in optimizing.

Also, what would be the appropriate way to assign bounds to parameters contained within x_guess?

The way I am approaching above issues:

I tried implementing the above constraint and populate the bounds for all parameters as shown below, but I get a keyword error coming from optunity's source code:

bounds = [(10, 1e+5),
 (10, 100000.0),
 (0.2, 0.8),
 (0.2, 0.8),
 (1000.0, 1e+10),
 (1000.0, 1e+10),
 (0, 1),
 (0, 1),
 (0, 1),
 (0, 1),
 (0, 1),
 (0, 1),
 (0, 1),
 (0, 1),
 (0, 1e-06)]

x_bounds = {'x_guess[{0}]'.format(i): [bounds[i][0], bounds[i][1]] for i in xrange(len(bounds))}

constraint = lambda x_guess: (1 - np.sum(x_guess))

min_fun = optunity.wrap_constraints(y, custom = [constraint]) # y = f(x)

optunity.minimize(min_fun, num_evals=100, solver_name = 'nelder-mead', **x_bounds)