search

jbrea / BayesianOptimization.jl

Bayesian optimization for Julia
Other
90 stars 17 forks source link

Allow GP to use given observation noise #5

Closed robsmith11 closed 5 years ago

robsmith11 commented 5 years ago

Thanks for sharing this package! I'm porting over some R code that I use to optimize parameters of a noisy simulation. I use DiceOptim/DiceKriging, which allows for a vector of observation variances to be passed (1 noise value for each observation). These explicit noise estimates can be used rather than estimating a nuggest effect with MLE.

Measuring the noise in this way is useful when performing multiple replications at a given set of input parameters. In addition to being a more reliable estimate of observation noise, it can also be used to reduce the Gaussian Process dimensionality by averaging the replications into a single observation ( n replications -> 1 mean and 1 variance).

I think most of the changes would need to be made in GaussianProcesses.jl, but wanted to check first that this approach made sense to you.

maximerischard commented 5 years ago

See this issue in GaussianProcesses.jl. Unfortunately, I don't think any of us have time to implement this feature right now, but I would be happy to point you in the right direction if you'd like to have a stab at implementing it yourself.

robsmith11 commented 5 years ago

@maximerischard Thanks. So you'd still recommend implementing it as masked vector noise kernel?

I tried playing around with the scalar noise kernel, but it wasn't obvious to me how to give it a fixed parameter (that optimize!() won't change). Is that not supported?

maximerischard commented 5 years ago

I've answered your first question in the other thread. To fix a parameter, use a FixedKernel: fix(noisekernel, :lσ) should do it. I'm really sorry this is undocumented :-(

robsmith11 commented 5 years ago

I'm going to close this for now as I've shifted my approach to modeling noise with a secondary GP using R's hetGP: https://cran.r-project.org/web/packages/hetGP/index.html

This approach handles the high degree of noise in my simulation results much better, but may take a bit of work to port over to Julia so I'll just use RCall for now.