Parameters dropped for compositional kernels

[PabloLeon]

I am trying to recreate the classic Mauna Loa example from Rasmussen(2006) [section 5.4.3 here http://www.gaussianprocess.org/gpml/chapters/RW5.pdf ].

import GPy
from sklearn.datasets import fetch_mldata

data = fetch_mldata('mauna-loa-atmospheric-co2').data
X = data[:, 1]
y = data[:, 0]

k1 = GPy.kern.RBF(1,variance=66.0, lengthscale=67.0)
k2 = GPy.kern.StdPeriodic(1, lengthscale=1.3, period=1.0) * GPy.kern.RBF(1, lengthscale=90.0, variance=2.4) 
k3 = GPy.kern.RatQuad(1,lengthscale=1.2,power=0.78, variance=0.66)
k4 = GPy.kern.RBF(1,lengthscale= 0.134, variance=0.18) + GPy.kern.White(1,variance=0.19**2)

k_rasmussen = k1 + k2 + k3 + k4

m_rasmussen = GPy.models.GPRegression(X[:,None],y[:,None],kernel=k_rasmussen,noise_var=1e-3)

Inspecting the model parameters with m_rasmussen, m_rasmussen.parameter_names() or m_rasmussen.kern drops the periodic component of kernel2. I only get: ['sum.rbf.variance', 'sum.rbf.lengthscale', 'sum.rbf_1.variance', 'sum.rbf_1.lengthscale', 'sum.white.variance', 'sum.RatQuad.variance', 'sum.RatQuad.lengthscale', 'sum.RatQuad.power', 'sum.sum.rbf.variance', 'sum.sum.rbf.lengthscale', 'sum.sum.white.variance', 'Gaussian_noise.variance'] If I set up the kernel as k1+k2+k3 the periodic is there.

[mzwiessele]

As a quick check: can you please try making the add kernel explicit:

k = GPy.kern.Add([k1.copy(),k2.copy(),k3.copy(),k4.copy()])

If that does not work, there is a bug.

On 28 Jul 2016, at 10:15, PabloLeon notifications@github.com wrote:

I am trying to recreate the classic Mauna Loa example from Rasmussen(2006) [section 5.4.3 here http://www.gaussianprocess.org/gpml/chapters/RW5.pdf ].

import GPy from sklearn.datasets import fetch_mldata

data = fetch_mldata('mauna-loa-atmospheric-co2').data X = data[:, 1] y = data[:, 0]

k1 = GPy.kern.RBF(1,variance=66.0, lengthscale=67.0) k2 = GPy.kern.StdPeriodic(1, lengthscale=1.3, period=1.0) * GPy.kern.RBF(1, lengthscale=90.0, variance=2.4) k3 = GPy.kern.RatQuad(1,lengthscale=1.2,power=0.78, variance=0.66) k4 = GPy.kern.RBF(1,lengthscale= 0.134, variance=0.18) + GPy.kern.White(1,variance=0.19**2)

k_rasmussen = k1 + k2 + k3 + k4

m_rasmussen = GPy.models.GPRegression(X[:,None],y[:,None],kernel=k_rasmussen,noise_var=1e-3) Inspecting the model parameters with m_rasmussen, m_rasmussen.parameter_names() or m_rasmussen.kern drops the periodic component of kernel2. I only get:

['sum.rbf.variance', 'sum.rbf.lengthscale', 'sum.rbf_1.variance', 'sum.rbf_1.lengthscale', 'sum.white.variance', 'sum.RatQuad.variance', 'sum.RatQuad.lengthscale', 'sum.RatQuad.power', 'sum.sum.rbf.variance', 'sum.sum.rbf.lengthscale', 'sum.sum.white.variance', 'Gaussian_noise.variance']

If I set up the kernel as k1+k2+k3 the periodic is there.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[PabloLeon]

@mzwiessele That works, thanks!

[mzwiessele]

The reason it does not work is quite technical. We try to keep the parts, without copy, so that you can adjust and watch them from the outside. That leads to unexpected behaviour. At some point we need to get rid is that and make copies of the kernels so that this does not happen anymore.

On 28 Jul 2016, at 11:06, PabloLeon notifications@github.com wrote:

@mzwiessele That works, thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[PabloLeon]

@mzwiessele The same then applies to mean functions?

m_fun_inter = GPy.mappings.Constant(1,1, value=-2696.8)
m_fun_slope = GPy.mappings.Linear(1,1)
m_fun_slope.A = 1.36
m_fun = GPy.mappings.Additive(m_fun_inter, m_fun_slope)

Now if I want to check the mean function m_fun returns empty and I can only see the constituents with m_fun.mapping1 etc... Wouldn't it be more natural to show the mean function constituents at the top (same applies to the models). E.g. show mapping.Additive as top level and then show the constituents with additive.mapping1,mapping2?

Related - It seems the mean function parameters are not optimized for compositions of mean functions? If I use the composition above and optimize the parameters stay fixed (even thought they are far from good for the data), if instead I only add the intercept it gets optimized. If this is a bug - should I attempt to fix it and create a pull request here or is this a bug in paramz?

[mzwiessele]

Good spotting, adding the parameters to the additive mean function was missing. This meant, that the parameters were not updated and not shown.

[mzwiessele]

The mappings are updated, looking at the combination kernel now : )