Open chenr86 opened 1 year ago
As per this paper:
"Since there is no analytical solution for estimating the hyperparameters θ, it is necessary to use numerical optimization to find the hyperparameters θ that maximize the likelihood function. This step is the most challenging in the construction of the kriging model. This is because, as previously mentioned, this estimation involves maximizing the likelihood function, which is often multimodal [Mardia and Watkins, 1989]. Maximizing this function becomes prohibitive for high- dimensional problems (d > 10) due to the cost of computing the determinant of the correlation matrix and the high number of evaluation needed for optimizing a high-dimensional multimodal problem."
Try p = 1.99 ones(dim) instead of p = 2 ones(dim). This will give better numerical stability (your covariance matrix is singular here, so its determinant is zero and that is why the logpdf is Inf).
@archermarx why 1.99
here? could you please explain more? Thanks!
When p = 2.0, the correlation function is nearly flat for points that are nearby, which causes ill conditioning in the correlation matrix. If p is just slightly less than 2.0, then the correlation function has a small slope so even points that are nearby are not counted as being effectively the same. An update I'm working on will change the default from 2.0 to 1.99
I have tried p = 1.99 * ones(dim). Still, the model is overfitting and there is nearly no improvement. The predictive coefficient R squared in training set is 1 while in test set is 0.00186.
Does your loss function still produce Inf?
Hello,
I am using surrogates.jl v6.3.0 to build kriging for a dataset with 500 elements. There are 24 input variables and 1 output. But I found the accuracy is terrible. There is serious overfitting with the model.
According to https://github.com/SciML/Surrogates.jl/issues/368, I calculate the loss function,
Inf
is obtained!I don't know how can I improve my model, as p and theta are already the values suggested.