After to many iterations in run of BayesianOptimization the error LinAlgError: not positive definite, even with jitter will be thrown

[Imdrail]

Several objective functions i testet worked quite normal for a small amount of iterations, but after some iterations the program crashes with an error message, which first says Model could not be trained, and at the end LinAlgError: not positive definite, even with jitter.

Here is some small code example which should reproduce this error(if not try with more iterations in opti.run)

import robo import GPy import math from robo.bayesian_optimization import BayesianOptimization from robo.models.GPyModel import GPyModel from robo.acquisition.EI import EI from robo.recommendation.incumbent import compute_incumbent from robo.maximizers.maximize import stochastic_local_search myX_upper = np.array([2.1]) myX_lower = np.array([-2.1])

objectiveFunction = lambda X: np.array([[-math.sin(X[0][0])-math.cos(2_X[0][0])+math.cos(9_X[0][0])]])

kernel = GPy.kern.RBF(input_dim=1, variance=30.1646253727, lengthscale=0.435343653946) noise = 1e-20 Model = GPyModel(kernel, noise_variance=noise, optimize=False)

ei_par = EI(Model, X_upper=myX_upper, X_lower=myX_lower, compute_incumbent=compute_incumbent, par=0.0)

opti = BayesianOptimization( acquisition_fkt=ei_par, model=Model, maximize_fkt=stochastic_local_search, X_lower=myX_lower, X_upper=myX_upper, dims=1, objective_fkt=objectiveFunction, save_dir=None, num_save=1)

initValues = opti.initialize()

newX = initValues[0] newY = initValues[1]

opti.run( num_iterations= 20 , X=newX, Y= newY, overwrite=True)

[aaronkl]

Your noise level is probably too hight, maybe set it to 1e-8. This is an issue with GPy which leads to numerical instabilities.

[voegtlel]

For me a similar issue was solved by using the newest version of GPy: pip install git+https://github.com/SheffieldML/GPy.git