'str' object has no attribute 'dim'

[JoKerDii]

I got ' 'str' object has no attribute 'dim'' by simply running the notebook. Any idea where is the error?

Epoch 1 / 10
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-31-c5138ddf6b25> in <module>
     12 
     13     #train model
---> 14     train_loss, _ = train()
     15 
     16     #evaluate model

<ipython-input-21-a8875e82e2a3> in train()
     25 
     26     # get model predictions for the current batch
---> 27     preds = model(sent_id, mask)
     28 
     29     # compute the loss between actual and predicted values

~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--> 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

<ipython-input-16-9ebdcf410f97> in forward(self, sent_id, mask)
     28       _, cls_hs = self.bert(sent_id, attention_mask=mask)
     29 
---> 30       x = self.fc1(cls_hs)
     31 
     32       x = self.relu(x)

~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--> 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

~/anaconda3/lib/python3.8/site-packages/torch/nn/modules/linear.py in forward(self, input)
     91 
     92     def forward(self, input: Tensor) -> Tensor:
---> 93         return F.linear(input, self.weight, self.bias)
     94 
     95     def extra_repr(self) -> str:

~/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py in linear(input, weight, bias)
   1686         if any([type(t) is not Tensor for t in tens_ops]) and has_torch_function(tens_ops):
   1687             return handle_torch_function(linear, tens_ops, input, weight, bias=bias)
-> 1688     if input.dim() == 2 and bias is not None:
   1689         # fused op is marginally faster
   1690         ret = torch.addmm(bias, input, weight.t())

AttributeError: 'str' object has no attribute 'dim'

[JoKerDii]

Figured it out by looking at this. It seems either specifying 'return_dict=False' or calling 'values()' method could force the bert model to return a tuple.

[SunilGundapu]

It's working with the older version. Try with transformers==3.0.2 version.

[JoKerDii]

Yeah, you are right, thanks!

[pul95]

@JoKerDii Did you specify return_dict=False where bert model was downloaded?

I mean

bert = AutoModel.from_pretrained('bert-base-uncased', return_dict=False)

[debanjandey64]

From this stackoverflow question,

After some 3.X version syntax in model architecture (forward pass) has got little changed.

Old code

pass the inputs to the model

  _, cls_hs = self.bert(sent_id, attention_mask=mask)
  x = self.fc1(cls_hs)

New code

pass the inputs to the model

  cls_hs = self.bert(sent_id, attention_mask=mask)
  x = self.fc1(cls_hs[1])