Bug about run a DGP model using py files in my platform instead of import gpytorch directly

[studying910]

Hi.

I am running a DGP model on my Jupyterlab using your tutorial providing on: https://docs.gpytorch.ai/en/stable/examples/05_Deep_Gaussian_Processes/Deep_Gaussian_Processes.html

The vanilla code can be used correctly. However, since I want to modlfy the code to realize my own IDEA, I have to download the code and import gpytorch using the code in my platform instead of import gpytorch directly. But it came with a bug as below:

---

TypeError Traceback (most recent call last)

in 17 with num_likelihood_samples(num_samples): 18 optimizer.zero_grad() ---> 19 output = model(x_batch) 20 loss = -mll(output, y_batch) 21 loss.backward() ~/dushian/gpytorch-master/gpytorch2/module.py in __call__(self, *inputs, **kwargs) 28 29 def __call__(self, *inputs, **kwargs): ---> 30 outputs = self.forward(*inputs, **kwargs) 31 if isinstance(outputs, list): 32 return [_validate_module_outputs(output) for output in outputs] in forward(self, inputs) 23 24 def forward(self, inputs): ---> 25 hidden_rep1 = self.hidden_layer(inputs) 26 output = self.last_layer(hidden_rep1) 27 return output in __call__(self, x, *other_inputs, **kwargs) 53 x = torch.cat([x] + processed_inputs, dim=-1) 54 ---> 55 return super().__call__(x, are_samples=bool(len(other_inputs))) ~/dushian/gpytorch-master/gpytorch2/models/deep_gps/deep_gp.py in __call__(self, inputs, are_samples, **kwargs) 100 if self.output_dims is not None: 101 mean = output.loc.transpose(-1, -2) --> 102 covar = BlockDiagLazyTensor(output.lazy_covariance_matrix, block_dim=-3) 103 output = MultitaskMultivariateNormal(mean, covar, interleaved=False) 104 ~/dushian/gpytorch-master/gpytorch/lazy/block_lazy_tensor.py in __init__(self, base_lazy_tensor, block_dim) ~/dushian/gpytorch-master/gpytorch/lazy/non_lazy_tensor.py in lazify(obj) TypeError: object of class SumLazyTensor cannot be made into a LazyTensor NOTE: I have modified the name of the file to 'gpytorch2' . Could you please tell me what is wrong with it? THX

[Balandat]

Sounds like some bug in your code? It’ll be hard to figure out what’s going wrong unless you can share the code that you’re running this on / provide a full repro.

[studying910]

Actually, I have not added any code. I just downloaded your code by 'git clone https://github.com/cornellius-gp/gpytorch'.

Then I run the ipynb file you provided in the tutorial: https://docs.gpytorch.ai/en/stable/examples/05_Deep_Gaussian_Processes/Deep_Gaussian_Processes.html

But it went wrong when I run the cell：

from gpytorch.settings import num_likelihood_samples num_epochs = 1 if smoke_test else 10 num_samples = 3 if smoke_test else 10

optimizer = torch.optim.Adam([ {'params': model.parameters()}, ], lr=0.01) mll = DeepApproximateMLL(VariationalELBO(model.likelihood, model, train_x.shape[-2])) # marginal log-likelihood

epochs_iter = tqdm.notebook.tqdm(range(num_epochs), desc="Epoch") for i in epochs_iter:

Within each iteration, we will go over each minibatch of data

minibatch_iter = tqdm.notebook.tqdm(train_loader, desc="Minibatch", leave=False)
for x_batch, y_batch in minibatch_iter:
    with num_likelihood_samples(num_samples):
        optimizer.zero_grad()
        output = model(x_batch)
        loss = -mll(output, y_batch)
        loss.backward()
        optimizer.step()

        minibatch_iter.set_postfix(loss=loss.item())

It seems like a bug when downloading your code to run on my own platform.

I am wondering if you can provide me any idea to solve this problem. THX :)

[studying910]

Sounds like some bug in your code? It’ll be hard to figure out what’s going wrong unless you can share the code that you’re running this on / provide a full repro.

The only thing I did was changing the name of 'gpytorch' to 'gpytorch2' and edit the code that import the file as follows:

%set_env CUDA_VISIBLE_DEVICES=0

import torch import tqdm from gpytorch2.means import ConstantMean, LinearMean from gpytorch2.kernels import RBFKernel, ScaleKernel from gpytorch2.variational import VariationalStrategy, CholeskyVariationalDistribution from gpytorch2.distributions import MultivariateNormal from gpytorch2.models import ApproximateGP, GP from gpytorch2.mlls import VariationalELBO, AddedLossTerm from gpytorch2.likelihoods import GaussianLikelihood from gpytorch2.models.deep_gps import DeepGPLayer, DeepGP from gpytorch2.mlls import DeepApproximateMLL

I also commented out 'import gpytorch' since I want to import the py file on my own platform so I could get access to it.

I had no idea about what' s wrong about the code.

THX :)