Alaya-in-Matrix
commented
2 years ago It looks like some acquisition function value calculated from Gaussian process model are NaN, this might be due to the numerical stability of GP model. Do you have a sample code that I can use to reproduce this crash?
Based on your design space, I assume you are tuning a XGBoost model, I wrote the below code using hebo.sklearn_tuner.sklearn_tuner to see if anything goes wrong, but it looks like everything was OK
import warnings
warnings.filterwarnings('ignore')
import numpy as np
import matplotlib.pyplot as plt
from hebo.optimizers.hebo import HEBO
from hebo.sklearn_tuner import sklearn_tuner
from hebo.design_space import DesignSpace
from xgboost import XGBRegressor
from sklearn.metrics import r2_score
from sklearn.model_selection import KFold, cross_val_predict
from sklearn.datasets import load_boston
space_cfg = [{'name': 'lr', 'type' : 'num', 'lb' : 0.00005, 'ub' : 0.1},
{'name': 'n_estimators', 'type' : 'int', 'lb' : 10, 'ub' : 200}, # multiplied by 10
{'name': 'max_depth', 'type' : 'int', 'lb' : 1, 'ub' : 10},
{'name': 'subsample', 'type' : 'num', 'lb' : 0.5, 'ub' : 0.99},
{'name': 'colsample_bytree', 'type' : 'num', 'lb' : 0.5, 'ub' : 0.99},
{'name': 'gamma', 'type' : 'num', 'lb' : 0.01, 'ub' : 10.0},
{'name': 'min_child_weight', 'type' : 'int', 'lb' : 1, 'ub' : 10},
{'name': 'fill_type', 'type' : 'cat', 'categories' : ['median', 'pat_median','pat_ema']},
{'name': 'flat_block_size', 'type' : 'int', 'lb' : 1, 'ub' : 1},
{'name': 'verbosity', 'type' : 'int', 'lb' : 0, 'ub' : 0}
]
X, y = load_boston(return_X_y = True)
cv = KFold(n_splits = 10, shuffle = True, random_state = 42)
result, report = sklearn_tuner(XGBRegressor, space_cfg, X, y, r2_score, cv = cv, max_iter = 64, report = True)
print(report)
report.metric.plot()
plot.show()
NotNANtoN
Hi, thanks for this repository! So far it works quite well, but now I suddenly encountered a weird error after 11 optimization steps of non-batched HEBO:
Seems like there is a NaN in some distribution of HEBO. But my input parameters (opt.X) and losses (opt.y) are never NaN. This is the design space I'm using:
I already commented out
flat_block_sizeas I thought that maybe it is a problem iflb == ub, but it still crashes.Any ideas on how I can debug this?