search

theogf / AugmentedGaussianProcesses.jl

Gaussian Process package based on data augmentation, sparsity and natural gradients
https://theogf.github.io/AugmentedGaussianProcesses.jl/dev/
Other
134 stars 9 forks source link

kernel hyperparameter learning #82

Open martinjankowiak opened 3 years ago

martinjankowiak commented 3 years ago

hello,

i'm confused about how to make sure that all the hyperparameters of my kernel are being learned.

in particular i would like to use a matern 5/2 kernel specified by D+1 parameters (D length scales and one kernel scale). to my understanding that can be specified as follows

kernel = transform(transform(Matern52Kernel(), ScaleTransform(1.0)), ARDTransform(ones(D)))

however, if i use Flux.params(kernel) to inspect the kernel hyperparameters after training it seems the hyperparameters haven't been updated.

note that i am using SVGP with the default optimiser so i would expect the hyperpararmeters to be updated. is this the wrong way to inspect the hyperparameters? do i need to do anything else to specify that i want the hyperparameters to be updated?

thank you!

theogf commented 3 years ago

Hi!

If you aim at having a kernel of the type k(x,y) = sigma * Matern52(X/lengthscale), ScaleTransform will not work, you can either use ScaledKernel(kernel, sigma) or more simply sigma * kernel.

About the lack of updates of the parameters, this should obviously not happen :). I will check what's happening. But note that a more stable version is currently worked on #71 but is waiting on https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/issues/203 to be solved.