huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
132k stars 26.29k forks source link

DebertaForSequenceClassification documents examples report RuntimeError: Index tensor must have the same number of dimensions as input tensor #9971

Closed johnson7788 closed 3 years ago

johnson7788 commented 3 years ago

Environment info

Who can help

Information

Model I am using (Bert, XLNet ...): Deberta

The problem arises when using:

The tasks I am working on is:

To reproduce

from transformers import DebertaTokenizer, DebertaForSequenceClassification
import torch
tokenizer = DebertaTokenizer.from_pretrained('microsoft/deberta-base')
model = DebertaForSequenceClassification.from_pretrained('microsoft/deberta-base')
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
labels = torch.tensor([1]).unsqueeze(0) 
outputs = model(**inputs, labels=labels)
loss = outputs.loss
logits = outputs.logits
print(loss)
print(logits)

Exception like below:

Traceback (most recent call last): File "/Users/admin/git/transformers/myexample4/deberta_MLM.py", line 65, in sequence_classify() File "/Users/admin/git/transformers/myexample4/deberta_MLM.py", line 45, in sequence_classify outputs = model(*inputs, labels=labels) File "/Users/admin/virtulenv/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(input, **kwargs) File "/Users/admin/git/transformers/src/transformers/models/deberta/modeling_deberta.py", line 1169, in forward labels = torch.gather(labels, 0, label_index.view(-1)) RuntimeError: Index tensor must have the same number of dimensions as input tensor

Process finished with exit code 1

Steps to reproduce the behavior:

1. 2. 3.

Expected behavior

NielsRogge commented 3 years ago

You shouldn't unsqueeze your labels, because the labels should just be a tensor of shape (batch_size,).