RF model plug-n-play broken

[cinaljess]

Hello,

In GPyOpt_reference_manual.ipynb (In 19, 20) I attempt to change the model_type from 'GP' to 'RF' for the sixhumpcamel example. myBopt2D = GPyOpt.methods.BayesianOptimization(f_sim.f, domain=bounds, model_type = 'RF', acquisition_type='EI',
normalize_Y = True, acquisition_weight = 2) # runs the optimization for the three methods max_iter = 40 # maximum time 40 iterations max_time = 60 # maximum time 60 seconds myBopt2D.run_optimization(max_iter,max_time,verbosity=False, save_models_parameters=False)

I added save_models_parameters=False to get past an exception, but then it seems to fail during the prediction method complaining of requiring a 2D array but only having a 1D array. Stack trace below:

[ 1.27904581  0.00736678]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-21-dcbedc0d1fb9> in <module>()
      3 max_time = 60  # maximum time 60 seconds
      4 
----> 5 myBopt2D.run_optimization(max_iter,max_time,verbosity=False, save_models_parameters=False)

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/core/bo.pyc in run_optimization(self, max_iter, max_time, eps, context, verbosity, save_models_parameters, report_file, evaluations_file, models_file)
    138                 break
    139 
--> 140             self.suggested_sample = self._compute_next_evaluations()
    141 
    142             # --- Augment X

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/core/bo.pyc in _compute_next_evaluations(self, pending_zipped_X, ignored_zipped_X)
    228 
    229         ### We zip the value in case there are categorical variables
--> 230         return self.space.zip_inputs(self.evaluator.compute_batch(duplicate_manager=duplicate_manager, context_manager= self.acquisition.optimizer.context_manager))
    231 
    232 

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/core/evaluators/sequential.pyc in compute_batch(self, duplicate_manager, context_manager)
     20         Selects the new location to evaluate the objective.
     21         """
---> 22         x, _ = self.acquisition.optimize(duplicate_manager=duplicate_manager)
     23         return x

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/acquisitions/base.pyc in optimize(self, duplicate_manager)
     55         """
     56         if not self.analytical_gradient_acq:
---> 57             out = self.optimizer.optimize(f=self.acquisition_function, duplicate_manager=duplicate_manager)
     58         else:
     59             out = self.optimizer.optimize(f=self.acquisition_function, f_df=self.acquisition_function_withGradients, duplicate_manager=duplicate_manager)

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/optimization/acquisition_optimizer.pyc in optimize(self, f, df, f_df, duplicate_manager)
     67 
     68         ## -- Select the anchor points (with context)
---> 69         anchor_points = anchor_points_generator.get(duplicate_manager=duplicate_manager, context_manager=self.context_manager)
     70 
     71         ## --- Applying local optimizers at the anchor points and update bounds of the optimizer (according to the context)

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/optimization/anchor_points_generator.pyc in get(self, num_anchor, duplicate_manager, unique, context_manager)
     57         X = X[non_duplicate_anchor_point_indexes,:]
     58 
---> 59         scores = self.get_anchor_point_scores(X)
     60 
     61         anchor_points = X[np.argsort(scores)[:min(len(scores),num_anchor)], :]

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/optimization/anchor_points_generator.pyc in get_anchor_point_scores(self, X)
     96     def get_anchor_point_scores(self, X):
     97 
---> 98         return self.objective(X).flatten()
     99 
    100 class RandomAnchorPointsGenerator(AnchorPointsGenerator):

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/acquisitions/base.pyc in acquisition_function(self, x)
     35         Takes an acquisition and weights it so the domain and cost are taken into account.
     36         """
---> 37         f_acqu = self._compute_acq(x)
     38         cost_x, _ = self.cost_withGradients(x)
     39         return -(f_acqu*self.space.indicator_constraints(x))/cost_x

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/acquisitions/EI.pyc in _compute_acq(self, x)
     34         Computes the Expected Improvement per unit of cost
     35         """
---> 36         m, s = self.model.predict(x)
     37         fmin = self.model.get_fmin()
     38         phi, Phi, u = get_quantiles(self.jitter, fmin, m, s)

/Users//anaconda2/lib/python2.7/site-packages/GPyOpt/models/rfmodel.py in predict(self, X)
     93             preds = []
     94             for pred in self.model.estimators_:
---> 95                 preds.append(pred.predict(X[k,:])[0])
     96             m = np.vstack((m ,np.array(preds).mean()))
     97             s = np.vstack((s ,np.array(preds).std()))

/Users//anaconda2/lib/python2.7/site-packages/sklearn/tree/tree.pyc in predict(self, X, check_input)
    410         """
    411         check_is_fitted(self, 'tree_')
--> 412         X = self._validate_X_predict(X, check_input)
    413         proba = self.tree_.predict(X)
    414         n_samples = X.shape[0]

/Users//anaconda2/lib/python2.7/site-packages/sklearn/tree/tree.pyc in _validate_X_predict(self, X, check_input)
    371         """Validate X whenever one tries to predict, apply, predict_proba"""
    372         if check_input:
--> 373             X = check_array(X, dtype=DTYPE, accept_sparse="csr")
    374             if issparse(X) and (X.indices.dtype != np.intc or
    375                                 X.indptr.dtype != np.intc):

/Users//anaconda2/lib/python2.7/site-packages/sklearn/utils/validation.pyc in check_array(array, accept_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
    408                     "Reshape your data either using array.reshape(-1, 1) if "
    409                     "your data has a single feature or array.reshape(1, -1) "
--> 410                     "if it contains a single sample.".format(array))
    411             array = np.atleast_2d(array)
    412             # To ensure that array flags are maintained

ValueError: Expected 2D array, got 1D array instead:
array=[ 1.27904582  0.00736678].
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.

Thank you,

[cinaljess]

My attempted fix is to change line 95 of rfmodel.py to encapsulate X[k, :] in a list: preds.append(pred.predict([X[k,:]])[0]) The RF model now runs but I have to evaluate if it runs correctly.

[takax-valefor]

I could not run the Gpyopt(which model is RF) too. Error message is "get_model_parametes() does not exist" Did you add "save_models_parameters=False" to get past this exception? where did you add? thank you.

[cinaljess]

I am not an author so in lieu of a best solution I'll offer a working one. You can add save_models_parameters=False to the line that runs optimization on your Bopt model. eg. myBopt2D.run_optimization(max_iter,max_time,verbosity=False, save_models_parameters=False)

[takax-valefor]

thank you for your reply. this information is useful for me.