dfm / george

Fast and flexible Gaussian Process regression in Python
http://george.readthedocs.io
MIT License
450 stars 128 forks source link

Scaling of multidimentional kernels #95

Open ranr01 opened 6 years ago

ranr01 commented 6 years ago

Hi,

When you multiply a kernel with a constant, $A$, you get an additional parameter to the kernel, $k_1$. However, I've noticed that this parameter is given by $k_1 = log(A/d)$ where d is the kernel's ndim (and not simply by $k_1=log(A))$).

Is this intentional? If so it might be a good idea to document it in the kernels section of the docs.

Below is a simple example.

Thanks, Ran

A simple example:

import george

kernel = 1.0 * george.kernels.Matern52Kernel(metric=[1,2],ndim=2)
gp = george.GP(kernel)

x = randn(3,2)
print(gp.get_matrix(x))

outputs something like:

[[1.         0.19238044 0.45855868]
 [0.19238044 1.         0.0575158 ]
 [0.45855868 0.0575158  1.        ]]

which is OK since we expect the diagonal to be 1.0.

However, if you look at the parameters:

params = gp.get_parameter_dict()
print(params)

you get:

OrderedDict([('kernel:k1:log_constant', -0.6931471805599453), ('kernel:k2:metric:log_M_0_0', 0.0), ('kernel:k2:metric:log_M_1_1', 0.6931471805599453)])

Where naively I would expect 'kernel:k1:log_constant' to be 0.0 and not log(1/2).

apsabelhaus commented 4 years ago

Bump, it just took me a bit to figure this out. It seems that @dfm answered this already in #57 and the intended result is indeed nominal/ndim.

This behavior doesn't appear to be documented (I'm happy to contribute a few lines of explanation if the maintainers would like), but more importantly, I'm unsure why george behaves this way. Shouldn't a ConstantKernel always be considered a scalar, even if ndim != 1, since kernels always output a scalar? What's the motivation for varying the hyperparameter according to ndim?

dfm commented 4 years ago

@apsabelhaus: This was chosen as an implementation detail really, but I understand that it is a bit strange! I don't have much time these days to commit, but I'd love to have a note in the docs if you have the time!

apsabelhaus commented 4 years ago

Thanks! Good to know. I'll figure out some documentation suggestions when I've got a few minutes. Until then, hopefully this issue points folks in the right direction.