Closed scmanjarrez closed 1 year ago
Hi @scmanjarrez
Thank you for pointing out this issue. I've just pushed a fix for it. It was a problem with the shape of the tensor representing the bias in one of the layers of the manually assembled GRU cell. It turns out, former versions of PyTorch (like former LTS version, 1.8.2) were more "forgiving" if you will, but the latest one surely isn't.
In PyTorch 1.8, we could do this:
>>> lin = nn.Linear(2, 2)
>>> lin.weight = nn.Parameter(torch.eye(2))
>>> lin.bias = nn.Parameter(torch.ones(1, 1, 2))
>>> x = torch.ones(1, 1, 2)
>>> lin(x)
tensor([[[2., 2.]]], grad_fn=<AddBackward0>)
But, in PyTorch 1.13 (which Colab is already using), the same code raises the exception you saw:
>>> lin = nn.Linear(2, 2)
>>> lin.weight = nn.Parameter(torch.eye(2))
>>> lin.bias = nn.Parameter(torch.ones(1, 1, 2))
>>> x = torch.ones(1, 1, 2)
>>> lin(x)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/dvgodoy/anaconda3/envs/pyt13/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/dvgodoy/anaconda3/envs/pyt13/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: expand(torch.FloatTensor{[1, 1, 2]}, size=[1, 2]): the number of sizes provided (2) must be greater or equal to the number of dimensions in the tensor (3)
>>> lin.bias = nn.Parameter(torch.ones(1, 2))
>>> lin(x)
tensor([[[2., 2.]]], grad_fn=<ViewBackward0>)
In the manually assembled GRU cell, the hidden state was multiplied by z
and added to the output (and I used it the bias of the linear layer to do that manually). The hidden state was a sequence of one following the (N=1,L=1,F=2) shape, so I simply used the first (and only) element of sequence instead, resulting in the necessary (1, 2) shape for the bias.
Best, Daniel
Hi, Figure22 is throwing the following error:
I have tested in local and colab, but the same error happens.