RuntimeError: One of the differentiated Tensors appears to not have been used in the graph.

[imirzadeh]

Hi,

I can't use this code to compute Hessian eigenvalues. Probably because I'm using the new version of Pytorch?
I'm using torch 1.4.0 with Python 3.6.9

Traceback (most recent call last):
  File "main.py", line 73, in <module>
    run()
  File "main.py", line 70, in run
    log_hessian(model, val_loader, time)
  File "main.py", line 28, in log_hessian
    eigenvals, eigenvecs = compute_hessian_eigenthings(model, loader, loss, num_eigenthings, use_gpu=use_gpu)
  File "/Users/iman/Work/research/nn-forget/hessian_eigenthings/hvp_operator.py", line 172, in compute_hessian_eigenthings
    hvp_operator, num_eigenthings, use_gpu=use_gpu, **kwargs
  File "/Users/iman/Work/research/nn-forget/hessian_eigenthings/power_iter.py", line 73, in deflated_power_iteration
    init_vec=prev_vec,
  File "/Users/iman/Work/research/nn-forget/hessian_eigenthings/power_iter.py", line 121, in power_iteration
    new_vec = operator.apply(vec) - momentum * prev_vec
  File "/Users/iman/Work/research/nn-forget/hessian_eigenthings/hvp_operator.py", line 49, in apply
    return self._apply_full(vec)
  File "/Users/iman/Work/research/nn-forget/hessian_eigenthings/hvp_operator.py", line 73, in _apply_full
    hessian_vec_prod = self._apply_batch(vec)
  File "/Users/iman/Work/research/nn-forget/hessian_eigenthings/hvp_operator.py", line 56, in _apply_batch
    grad_vec = self.prepare_grad()
  File "/Users/iman/Work/research/nn-forget/hessian_eigenthings/hvp_operator.py", line 109, in prepare_grad
    loss, self.model.parameters(), create_graph=True
  File "/Users/iman/.pyenv/versions/opt_transfer/lib/python3.6/site-packages/torch/autograd/__init__.py", line 157, in grad
    inputs, allow_unused)
RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.

[imirzadeh]

I tried with Pytorch 0.4.0 and I still get the error.
Apparently the problem is with cross entropy loss. Because I can run the tests without error with MSE Loss.

[imirzadeh]

Okay. My bad. I had some modules that I was not using for forward pass. For example, below I had nn.batchnorm, but I wasn't using for froward pass! But when I used loss_grad = torch.autograd.grad(loss, model.parameters(), create_graph=True), the model.parameters() had batchnorms.

class MLP(nn.Module):
    def __init__(self, hidden_layers, config):
        super(MLP, self).__init__()
        self.W1 = nn.Linear(784, hidden_layers[0])
        self.relu = nn.ReLU(inplace=True)
            .....
        self.batchnorm = nn.BatchNorm1d(hidden_layers[0])

       def foward(self, x):
          out  = self.relu(self.W1(x)) #no batchnorm!