apaleyes
commented
1 year ago Hi! The question you are asking probably doesn't have a great answer. Lengthscale is a parameter, and if its poorly fit the model will not perform well. Likewise if you trained your model to great performance, you parameters probably have good values. How are you ensuring the quality of your final model after training?
Couple of general thoughts. First, the prior value of 15 - where does this come from? It might be very relevant, but it usually requires some sort of knowledge about your function. Here is a good answer that gives some intuition about influence of lengthscale.
Second, at the moment your code assumes one lengthscale for all dimensions of the input. If that's not desired, consider setting ARD=True to train separate lengthscale per input parameter.
Third, have a look at the output lengthscales. If they are really big numbers (orders of magnitude bigger than the scale of the input), this means the GP can't learn anything. Although this will also be obvious because the model will just become flat prior everywhere. If lengthscales are of reasonable values, and you used ARD, compare them. Smaller lengthscale usually means the input feature is more important. Do the lengthscale values you observe match your intuition about the inputs?
No definite answer here, but instead a bunch of hints. Hope you find this helpful!
eafshari
Hello, I am using the Emukit for my problem. I have a 3D probem (2 independent input and 1 output) and I obtained the response surface by using linear multi-fidelity modeling in Emukit. But I don't know that my lengthscales have been optimized correctly or not. Could you help me please? here is my code: X_L = np.array([[0.1,60],[0.1,55],[0.1,50],[0.1,45],[0.1,40],[0.1,35],[0.1,30],[0.1,25],[0.1,20],[0.1,15],[0.1,10], [0.2,60],[0.2,55],[0.2,50],[0.2,45],[0.2,40],[0.2,35],[0.2,30],[0.2,25],[0.2,20],[0.2,15],[0.2,10], [0.3,60],[0.3,55],[0.3,50],[0.3,45],[0.3,40],[0.3,35],[0.3,30],[0.3,25],[0.3,20],[0.3,15],[0.3,10] ]) y_L = np.array([0.2254,0.2273,0.2309,0.2351,0.2384,0.24,0.2392,0.2554,0.2615,0.2601,0.2551, 0.1795,0.1793,0.1806,0.1825,0.1842,0.1852,0.1854,0.1975,0.2036,0.2066,0.2107, 0.1428,0.1411,0.1409,0.1415,0.1427,0.1443,0.1466,0.1569,0.1651,0.1740,0.1877 ])
X_H = np.array([[0.33, 50], [0.33, 40], [0.33, 35], [0.33, 30], [0.45, 50], [0.45, 40], [0.45, 35], [0.45, 30], [0.6, 50], [0.6, 40], [0.6, 35], [0.6, 30], [0.73, 50], [0.73, 40], [0.73, 35], [0.73, 30] ]) y_H = np.array([0.12401, 0.149385, 0.174739, 0.20918, 0.0960703, 0.0879624, 0.108555, 0.141227, 0.0591088, 0.0613414, 0.0669841, 0.0965032, 0.0559802, 0.0570884, 0.0597303, 0.0734654 ])
x_train_l = np.atleast_2d(X_L) x_train_h = np.atleast_2d(X_H) y_train_h = np.atleast_2d(y_H).T y_train_l = np.atleast_2d(y_L).T
Convert lists of arrays to ndarrays augmented with fidelity indicators
from emukit.multi_fidelity.convert_lists_to_array import convert_x_list_to_array, convert_xy_lists_to_arrays
X_train, Y_train = convert_xy_lists_to_arrays([x_train_l, x_train_h], [y_train_l, y_train_h])
Construct a linear multi-fidelity model
kernels = [GPy.kern.RBF(input_dim=2,lengthscale=15), GPy.kern.RBF(input_dim=2,lengthscale=15)] lin_mf_kernel = emukit.multi_fidelity.kernels.LinearMultiFidelityKernel(kernels) gpy_lin_mf_model = GPyLinearMultiFidelityModel(X_train, Y_train, lin_mf_kernel, n_fidelities=2) gpy_lin_mf_model.mixed_noise.Gaussian_noise.fix(0.0001) gpy_lin_mf_model.mixed_noise.Gaussian_noise_1.fix(0.0001)
Wrap the model using the given 'GPyMultiOutputWrapper'
lin_mf_model = model = GPyMultiOutputWrapper(gpy_lin_mf_model, 2, n_optimization_restarts=10)
Fit the model
lin_mf_model.optimize()
My problem is I entered the lengthscales for each dimension 15 and after the prediction, I saw the lengthscales from this code: lin_mf_model.gpy_model.kern. But by changing the initial lengthscales, I obtained different response surfaces. How can I be sure my lenghscales are optimized correctly?