Outdated reinforcement_q_learning.py tutorial - throwing several warnings and errors because of "volatile"

[sahiljuneja]

https://github.com/pytorch/tutorials/blob/master/intermediate_source/reinforcement_q_learning.py

Some of the errors I get -

/usr/local/lib/python2.7/dist-packages/torchvision-0.2.0-py2.7.egg/torchvision/transforms/transforms.py:176: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
/usr/lib/python2.7/dist-packages/matplotlib/backend_bases.py:2437: MatplotlibDeprecationWarning: Using default event loop until function specific to this GUI is implemented
  warnings.warn(str, mplDeprecation)
gym-DQN.py:335: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  Variable(state, volatile=True).type(FloatTensor)).data.max(1)[1].view(1, 1)
gym-DQN.py:398: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  volatile=True)
gym-DQN.py:413: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
  next_state_values.volatile = False
Traceback (most recent call last):
  File "gym-DQN.py", line 466, in <module>
    optimize_model()
  File "gym-DQN.py", line 418, in optimize_model
    loss = F.smooth_l1_loss(state_action_values, expected_state_action_values)
RuntimeError: the derivative for 'target' is not implemented

Trying to fix them myself, but I am new to pytorch. Any other references for this code?

[soumith]

this happens if you use PyTorch's master branch. the tutorial is kept up to date for the latest release which is pytorch v0.3.0

[sahiljuneja]

Oh ok. Thanks for clarifying. I tried to install 0.3.0 as well which led to it trying to uninstall 0.4.0 but ran into an error.

Found existing installation: torch 0.4.0a0+7592e96
    Can't uninstall 'torch'. No files were found to uninstall.
  Can't roll back torch; was not uninstalled
Exception:
Traceback (most recent call last):
  File "/home/sahil/.local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/sahil/.local/lib/python2.7/site-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/home/sahil/.local/lib/python2.7/site-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/home/sahil/.local/lib/python2.7/site-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/home/sahil/.local/lib/python2.7/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/home/sahil/.local/lib/python2.7/site-packages/pip/wheel.py", line 345, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/home/sahil/.local/lib/python2.7/site-packages/pip/wheel.py", line 316, in clobber
    ensure_dir(destdir)
  File "/home/sahil/.local/lib/python2.7/site-packages/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/torch-0.3.0.post4.dist-info'

This has something to do with an alternative repo which installed it from source I think. I will look into this error and see how to get rid of it and install 0.3.0 again. [Unless, you already have an answer to solve the above]

Thanks for the quick reply! Will close the issue.