TypeError: iteration over a 0-d tensor

[philtomson]

I tried running:

python main.py --network_type rnn --dataset ptb --controller_optim adam --controller_lr 0.00035 --shared_optim sgd --shared_lr 20.0 --entropy_coeff 0.0001

But got:

2018-09-20 11:34:56,560:INFO::[*] Make directories : logs/ptb_2018-09-20_11-34-56
2018-09-20 11:35:01,015:INFO::regularizing:
2018-09-20 11:35:06,032:INFO::# of parameters: 146,014,000
2018-09-20 11:35:06,127:INFO::[*] MODEL dir: logs/ptb_2018-09-20_11-34-56
2018-09-20 11:35:06,128:INFO::[*] PARAM path: logs/ptb_2018-09-20_11-34-56/params.json
/home/phil/anaconda3/envs/conda_env/lib/python3.7/site-packages/torch/nn/functional.py:995: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
  warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
/home/phil/anaconda3/envs/conda_env/lib/python3.7/site-packages/torch/nn/functional.py:1006: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.
  warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.")
Traceback (most recent call last):
  File "main.py", line 48, in <module>
    main(args)
  File "main.py", line 34, in main
    trnr.train()
  File "/home/phil/devel/ENAS-pytorch/trainer.py", line 216, in train
    self.train_shared()
  File "/home/phil/devel/ENAS-pytorch/trainer.py", line 297, in train_shared
    hidden = utils.detach(hidden)
  File "/home/phil/devel/ENAS-pytorch/utils.py", line 130, in detach
    return tuple(detach(v) for v in h)
  File "/home/phil/devel/ENAS-pytorch/utils.py", line 130, in <genexpr>
    return tuple(detach(v) for v in h)
  File "/home/phil/devel/ENAS-pytorch/utils.py", line 130, in detach
    return tuple(detach(v) for v in h)
  File "/home/phil/devel/ENAS-pytorch/utils.py", line 130, in <genexpr>
    return tuple(detach(v) for v in h)
  File "/home/phil/devel/ENAS-pytorch/utils.py", line 130, in detach
    return tuple(detach(v) for v in h)
  File "/home/phil/anaconda3/envs/conda_env/lib/python3.7/site-packages/torch/tensor.py", line 381, in __iter__
    raise TypeError('iteration over a 0-d tensor')
TypeError: iteration over a 0-d tensor

Running Pytorch 0.4.1 on Python 3.7 (also tried on python 3.6.6, pytorch 0.4.1 and had same issue)

[philtomson]

debugging this a bit, it calls utils.detatch() and ends up at:

-> return tuple(detach(v) for v in h)
  /home/phil/devel/ENAS-pytorch/utils.py(131)detach()
-> return tuple(detach(v) for v in h)
> /home/phil/.virtualenvs/p3.6/lib/python3.6/site-packages/torch/tensor.py(380)__iter__()
-> if self.dim() == 0:
(Pdb) pp self
tensor(0.2035, device='cuda:0', grad_fn=<SelectBackward>)
(Pdb) p self.dim()
0

So since it's a scalar and the dim is 0 it's going to raise a type error:

(Pdb) ll
373         def __iter__(self):
374             # NB: we use 'imap' and not 'map' here, so that in Python 2 we get a
375             # generator and don't eagerly perform all the indexes.  This could
376             # save us work, and also helps keep trace ordering deterministic
377             # (e.g., if you zip(*hiddens), the eager map will force all the
378             # indexes of hiddens[0] before hiddens[1], while the generator
379             # map will interleave them.)
380  ->         if self.dim() == 0:
381                 raise TypeError('iteration over a 0-d tensor')

[philtomson]

If I'm understanding the response to this issue: https://github.com/carpedm20/ENAS-pytorch/issues/22 PyTorch 0.4.1 is not supported (needs to be 0.3.1?)

[philtomson]

Just an FYI for folks wanting to run this code under versions of Pytorch >= 0.4.0: In trainer.py you should not use the call to utils.detach() - instead you should call detach_() on hidden:

Change from:

hidden = utils.detach(hidden)

to:

hidden.detach_()

[usmanhidral]

I resolved this error by changing that function in lstm_rnn fastai file Before def repackage_var(h): return Variable(h.data) if type(h) == Variable else tuple(repackage_var(v) for v in h) After def repackage_var(h): if not isinstance(h, list) and not isinstance(h, tuple) and len(h.size())==0: h = torch.tensor([[h.data]]) return Variable(h.data) if type(h) == Variable else tuple(repackage_var(v) for v in h)