Multi-metric scoring with non-default `search_optimization` values

[mattKretschmer]

When using multi-metric scoring with non-default values for search_optimization in TuneSearchCV, I was getting a KeyError when the refitting process was happening. I've included a code snipped at a screenshot of the error below. I had already successfully installed scikit-optimize and hyperopt already.

A code-snipped that reproduces this error is:

from tune_sklearn import TuneSearchCV
import numpy as np
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from xgboost import XGBRegressor

# Set training and validation sets
X, y = make_regression(n_samples=100, n_features=10, n_informative=5)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=20)

# Example parameters to tune from xgb
parameters = {
                "regressor__max_depth": [2,3,5,8],
                "regressor__learning_rate": (0.05, 0.2, 'uniform')
}

tune_search = TuneSearchCV(
    XGBRegressor(random_state=42, objective='reg:squarederror',
                n_estimators=1000, verbose=False, n_jobs = 1),
    parameters,
    n_jobs=1,
    refit='mae',
    search_optimization='bayesian',
    n_trials = 2,
    scoring = {
                "mae": "neg_mean_absolute_error",
                "rmse": "neg_root_mean_squared_error",
                "r2": "r2",
    }
)
tune_search.fit(X_train, y_train)
end = time.time()

[richardliaw]

Closed by #111 !