search

cardiffnlp / xlm-t

Repository for XLM-T, a framework for evaluating multilingual language models on Twitter data
Apache License 2.0
145 stars 23 forks source link

Use models with zero-shot classification #3

Open kinoute opened 3 years ago

kinoute commented 3 years ago

Hello,

First, thanks for these great models! I was wondering if I could use these models for zero-shots classification, especially for emotion detection (Ekman). While doing so, I encountered this error regarding the config of the models and the missing entailment parameter which seems mandatory for zero-shot classification:

Failed to determine 'entailment' label id from the label2id mapping in the model config. Setting to -1. Define a descriptive label2id mapping in the model config to ensure correct outputs.

Basic code to reproduce:

model_path = "cardiffnlp/twitter-xlm-roberta-base"
zero_shot = pipeline("zero-shot-classification", model=model_path, tokenizer=model_path,)
zero_shot("You are an awful man,", candidate_labels=["anger", "disgust", "fear", "joy", "sadness", "surprise"], multi_label=True,)

It seems to lead indeed to weird outputs compared to the XLM-Roberta XNLI model that can be tested here: https://huggingface.co/zero-shot/.

Is there something missing in head_config.json?

Thanks!

njfm0001 commented 2 years ago

I was wondering the same. Any news?