Closed gwc4github closed 3 years ago
Hi,
Tokenizers in HuggingFace Transformers don't take care of padding labels (this should be done by the user). You can only provide text to a tokenizer, and it will turn them into input_ids
, attention_mask
and token_type_ids
. The tokenize_and_align_labels
function will take care of labeling each token.
Thanks for this note @NielsRogge and sorry for the delay getting back to you. Lots to do here. We will make the change in our code but it seems like this would be a good feature for the framework and the code is done.
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
@NielsRogge can we at least get a better error message for this?
hello how can i find acceptable labels for train_data to fine tuning a pretrained transformer sentiment model>?
Environment info
transformers
version: 4.6.1Who can help
@LysandreJik
Information
Model I am using (Bert, XLNet ...): Bert
The problem arises when using:
The tasks I am working on is:
To reproduce
Steps to reproduce the behavior:
Expected behavior
I am expecting it to train the model on our custom data. It was failing during training and I found the bug and fixed it. So mostly I am just trying to report the bug.
The bug is in transformers/tokenization_utils_base.py at line 2990. In the _pad() method you forgot to add an if statement for labels. More specifically you have a if self.padding_side == "right": and a if self.padding_side == "left": and both of them are missing the nested if for labels. (The have one for token_type_ids & special_tokens_mask)
You should add the section for both left and right but here is the change I made for the "right" side: