RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [256, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead.

[pengzhenghao]

Hi there!

Thanks for open-sourcing this work! Congratulation to SUNRISE, that's a great work!

I am trying to run SUNRISE in Walker2d-v3 environment. I have modified the get_env function to use gym.make("Walker2d-v3").

But I encounter the following error:

Traceback (most recent call last):
  File "sunrise.py", line 203, in <module>
    experiment(variant)
  File "sunrise.py", line 156, in experiment
    algorithm.train()
  File "/home/pengzhenghao/drivingforce/drivingforce/dice/sunrise/rlkit/core/rl_algorithm.py", line 46, in train
    self._train()
  File "/home/pengzhenghao/drivingforce/drivingforce/dice/sunrise/rlkit/core/batch_rl_algorithm.py", line 82, in _train
    self.trainer.train(train_data)
  File "/home/pengzhenghao/drivingforce/drivingforce/dice/sunrise/rlkit/torch/torch_rl_algorithm.py", line 50, in train
    self.train_from_torch(batch)
  File "/home/pengzhenghao/drivingforce/drivingforce/dice/sunrise/rlkit/torch/sac/neurips20_sac_ensemble.py", line 260, in train_from_torch
    policy_loss.backward()
  File "/home/pengzhenghao/anaconda3/envs/pgdrive/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/pengzhenghao/anaconda3/envs/pgdrive/lib/python3.7/site-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [256, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

Do you have any suggestion on that? The pytorch version listed in the requirement is too old. I am using the latest pytorch now. Does the error relate to the pytorch version?

Thanks!

[pokaxpoka]

Looks like a version issue but 100% sure...

[sweetice]

But I really don't think this is not a bug.

[WangYong-Design]

I also encountered this problem, how did you solve it?