Can this be used for hyper-parameters?

[kroscek]

Hi. Can stochOpy be used for hyper-parameters just like in the Bayesian-Optimization package? For example I want to find the optimal C parameter from SVR regression sklearn that can minimize MAE objective function. So the example is as follows:

dicts={'params':{'C':[-1e3,1e3]}}
def MAE_func(**param):
    model.set_params(**param)
    model.fit(train, y_train)

cv = Evolutionary(MAE_func,lower=lower,upper=upper,max_iter = 200,snap = True)
xopt, gfit = cv.optimize(solver = "cpso")

But it returns an error:

File "/home/lemma/train.py", line 1998, in optimize_genetic1 xopt, gfit = cv.optimize(solver = "cpso") File "/home/lemma/Evo.py", line 279, in optimize xstart = xstart, sync = sync) File "/home/lemma/Evo.py", line 666, in _cpso pfit = self._eval_models(X) File "/home/lemma/Evo.py", line 306, in _eval_models fit_mpi[i] = self._func(self._unstandardize(models[i,:])) File "/home/lemma/Evo.py", line 78, in self._func = lambda x: func(x, *args, **kwargs) TypeError: fun() takes exactly 0 arguments (1 given)

Not sure what went wrong with this?

[keurfonluu]

Hi @wa1618i,

The parameter you are trying to optimize is the damping coefficient C. Assuming 'model' is defined as your SVR estimator, your cost function should be declared this way:

def MAE_func(val):
    model.set_params(C = val)
    model.fit(train, y_train)
    # rest of the code
    return -best_score

Also, as you have only one parameter to optimize, 'lower' and 'upper' must be lists of length 1, therefore:

lower = [ left ]
upper = [ right ]

This should do the trick. However, using an evolutionary algorithm to solve this problem may not be suitable. It will work, but I think a simple grid search or a golden section search is more appropriate for this problem.

Hope it helps!