SamLynnEvans / Transformer

Transformer seq2seq model, program that can build a language translator from parallel corpus
Apache License 2.0
1.35k stars 350 forks source link

AttributeError: 'int' object has no attribute 'dim' #22

Closed askaydevs closed 4 years ago

askaydevs commented 4 years ago

I was following your post on Medium, thanks for the great walkthrough. While training the model I landed an error, following is the traceback of the error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-28-2c3e6665094d> in <module>
----> 1 train_model(1)

<ipython-input-22-ed997c68c1f0> in train_model(epochs, print_every)
     22             # create function to make masks using mask code above
     23             src_mask, trg_mask = create_masks(src, trg_input)
---> 24             preds = model(src, trg_input, src_mask, trg_mask)
     25 
     26             optim.zero_grad()

~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

<ipython-input-19-c19adc4fb35e> in forward(self, src, trg, src_mask, trg_mask)
      7 
      8     def forward(self, src, trg, src_mask, trg_mask):
----> 9         e_outputs = self.encoder(src, src_mask)
     10         d_output = self.decoder(trg, e_outputs, src_mask, trg_mask)
     11         output = self.out(d_output)

~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

<ipython-input-18-75ae85de9dd0> in forward(self, src, mask)
     12         x = self.pe(x)
     13         for i in range(N):
---> 14             x = self.layers[i](x, mask)
     15         return self.norm(x)
     16 

~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

<ipython-input-17-1d977cf7013c> in forward(self, x, mask)
     13     def forward(self, x, mask):
     14         x2 = self.norm_1(x)
---> 15         x = x + self.dropout_1(self.attn(x2, x2, 2, mask))
     16         x2 = self.norm_2(x)
     17         x = x + self.dropout_2(self.ff(x2))

~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

<ipython-input-14-ead1732683eb> in forward(self, q, k, v, mask)
     34         k = self.k_linear(k).view(bs, -1, self.h, self.d_k)
     35         q = self.q_linear(q).view(bs, -1, self.h, self.d_k)
---> 36         v = self.v_linear(v).view(bs, -1, self.h, self.d_k)
     37 
     38         # transpose to get dimensions bs * h * sl * d_model

~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/torch/nn/modules/linear.py in forward(self, input)
     85 
     86     def forward(self, input):
---> 87         return F.linear(input, self.weight, self.bias)
     88 
     89     def extra_repr(self):

~/anaconda3/envs/tf-chatbot/lib/python3.6/site-packages/torch/nn/functional.py in linear(input, weight, bias)
   1606         if any([type(t) is not Tensor for t in tens_ops]) and has_torch_function(tens_ops):
   1607             return handle_torch_function(linear, tens_ops, input, weight, bias=bias)
-> 1608     if input.dim() == 2 and bias is not None:
   1609         # fused op is marginally faster
   1610         ret = torch.addmm(bias, input, weight.t())

AttributeError: 'int' object has no attribute 'dim'

I am unable to solve this as I am new to PyTorch. Pl, help me solve the issue.

askaydevs commented 4 years ago

Solved!