gpleiss
commented
3 years ago I cannot reproduce your error based on the code example you provide. I had to fix the code example to get it to run:
import gpytorch
import numpy as np
import torch
class MultitaskGPModel(gpytorch.models.ApproximateGP):
def __init__(self):
inducing_points = Zc_intra
variational_distribution = gpytorch.variational.CholeskyVariationalDistribution(
inducing_points.size(-2), batch_shape=torch.Size([num_latents])
)
variational_strategy = gpytorch.variational.LMCVariationalStrategy(
gpytorch.variational.VariationalStrategy(
self, inducing_points, variational_distribution, learn_inducing_locations=True
),
num_tasks=num_tasks,
num_latents=num_latents,
# could be 0
latent_dim=-1
)
super().__init__(variational_strategy)
self.mean_module = gpytorch.means.ConstantMean(batch_shape=torch.Size([num_latents]))
self.covar_module = gpytorch.kernels.ScaleKernel(
gpytorch.kernels.PeriodicKernel(batch_shape=torch.Size([num_latents])) *
gpytorch.kernels.RQKernel(batch_shape=torch.Size([num_latents])) +
gpytorch.kernels.ScaleKernel(
gpytorch.kernels.MaternKernel(nu=0.5, batch_shape=torch.Size([num_latents])),
batch_shape=torch.Size([num_latents])),
batch_shape=torch.Size([num_latents])
)
def forward(self, x):
mean_x = self.mean_module(x)
covar_x = self.covar_module(x)
return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)
Zc_intra_np = np.arange(0, 24, 1).reshape(-1, 1)
Zc_intra = torch.tensor(Zc_intra_np, dtype=torch.float)
w_l = 50
num_latents = 24
num_tasks = 12
Xc_t_npa = np.arange(0,w_l,1,dtype=np.float32).reshape(-1, 1)
Xc_t = torch.from_numpy(Xc_t_npa).type(torch.Tensor)
model_mul = MultitaskGPModel()
likelihood_mul = gpytorch.likelihoods.MultitaskGaussianLikelihood(num_tasks=num_tasks)
model_mul(Xc_t).covariance_matrix
lucheroni
🐛 Bug: Possible error with multitask learning with additive kernel structure
When I define in the class MultitaskGPModel the multitask kernel
which uses the additive kernel as its outermost layer, and I apply the class on data as
w_l = 50 num_latents = 24 Xc_t_npa = np.arange(0,w_l,1,dtype=np.float32).reshape(-1, 1) Xc_t = torch.from_numpy(Xc_t_npa).type(torch.Tensor) model_mul(Xc_t)
I get 'RuntimeError: The expected shape of the kernel was torch.Size([100, 100]), but got torch.Size([24, 100, 100]). This is likely a bug in GPyTorch'. This behavior seems not to change when changing the number of tasks or the number of latent gps.
If I use the same kernel in a non-batch setting, it works smoothly.
I wrote the batched problem with another kernel which is mathematically the same but which doesn't use the outer additive kernel, and it works smoothly. Unfortunatly the role of the subkernel parameters in the new form is not the same as that of the malfunctioning kernel, and I have to re-run a lot of past non-batch fits in the new form to make them comparable with the new setting.
To reproduce
Code snippet to reproduce
Stack trace/error message
Expected Behavior
Run with no errors ## System information **Please complete the following information:** - 1.4.1 - 1.8.1 - Win10 pro 19042.1052 ## Additional context Add any other context about the problem here.