[Bug] Error in the "initialize" method of BatchIndependentMultitaskGPModel.

[Miguel-Hombrados]

🐛 Bug

When the method "initialize" is used passing a dictionary of the parameters, in the case of an additive kernel. In particular, I suspect "_get_module_and_name()" is not splitting correctly the dictionary keys into modules list, that is why is not finding it within the ModuleList

Code snippet to reproduce

Iinitalization of the parameters
 hypers = {
            'likelihood.noise': torch.tensor(1.),
            'likelihood.task_noises': 0.1*torch.ones(task_num),
            'covar_module.kernels[0].base_kernel.outputscale': 0.1*torch.ones(task_num)
            'covar_module.kernels[1].bias': 1*torch.ones(task_num,1,1)   # Scale Kernel
            }
Kernel definition:
         kernel_cov =  gpytorch.kernels.RBFKernel(batch_shape=torch.Size([num_task]))
        self.covar_module = gpytorch.kernels.ScaleKernel(kernel_cov,
            batch_shape=torch.Size([num_task])
        ) + bias(batch_shape=torch.Size([num_task])) 
# Please make sure it does not require any external dependencies (other than PyTorch!)
# (We much prefer small snippets rather than links to existing libraries!)

Stack trace/error message

"Invalid parameter name {}. {} has no module {}".format(parameter_name, type(self).__name__, module)

AttributeError: Invalid parameter name kernels[0].base_kernel.outputscale. AdditiveKernel has no module kernels[0]

Expected Behavior

Proper initialization of the parameters of the sum of kernels.

System information

Please complete the following information:

• GPytorch 1.6.0
• Pytorch 1.10.0+cpu
• Windows 10 Pro

Additional context

The second kernel added is a custom kernel, but that should not be a problem since the error occurs before that.

[wjmaddox]

Can you provide a model class here? From what I see, the tutorial doesn't include the additive kernel portion so it'd be nice to see what exactly is going wrong.

[Miguel-Hombrados]

The kernel is defined as:

kernel_cov = gpytorch.kernels.LinearKernel(batch_shape=torch.Size([num_task])) self.covar_module = gpytorch.kernels.ScaleKernel(kernel_cov, batch_shape=torch.Size([num_task]) ) + bias(batch_shape=torch.Size([num_task])) The class of the custom kernel is:

`class bias(gpytorch.kernels.Kernel):

the sinc kernel is stationary

is_stationary = False

# We will register the parameter when initializing the kernel
def __init__(self, bias_prior=None, bias_constraint=None, **kwargs):
    super().__init__(**kwargs)

    # register the raw parameter
    self.register_parameter(
        name='raw_bias', parameter=torch.nn.Parameter(torch.zeros(*self.batch_shape, 1, 1))
    )

    # set the parameter constraint to be positive, when nothing is specified
    if bias_constraint is None:
        bias_constraint = Positive()

    # register the constraint
    self.register_constraint("raw_bias", bias_constraint)

    # set the parameter prior, see
    # https://docs.gpytorch.ai/en/latest/module.html#gpytorch.Module.register_prior
    if bias_prior is not None:
        self.register_prior(
            "bias_prior",
            bias_prior,
            lambda m: m.bias,
            lambda m, v : m._set_bias(v),
        )

# now set up the 'actual' paramter
@property
def bias(self):
    # when accessing the parameter, apply the constraint transform
    return self.raw_bias_constraint.transform(self.raw_bias)

@bias.setter
def bias(self, value):
    return self._set_bias(value)

def _set_bias(self, value):
    if not torch.is_tensor(value):
        value = torch.as_tensor(value).to(self.raw_bias)
    # when setting the paramater, transform the actual value to a raw one by applying the inverse transform
    self.initialize(raw_bias=self.raw_bias_constraint.inverse_transform(value))

# this is the kernel function
def forward(self, x1, x2, **params):
    # apply biasscale
    x1_ = x1
    x2_ = x2
    # calculate the distance between inputs
    diff = self.covar_dist(x1_, x2_, **params)
    diff0 = torch.ones_like(diff)*self.bias
    # prevent divide by 0 errors
    diff0.where(diff0 == 0, torch.as_tensor(1e-20))
    # return sinc(diff) = sin(diff) / diff
    return diff0`