Unable to get predictions for multiple sentences using TARS Zero Shot Classifier

[nlproc-007]

Here is the example code to use TARS Zero Shot Classifier

from flair.models import TARSClassifier
from flair.data import Sentence

# 1. Load our pre-trained TARS model for English
tars = TARSClassifier.load('tars-base')

# 2. Prepare a test sentence
sentence = Sentence("I am so glad you liked it!")

# 3. Define some classes that you want to predict using descriptive names
classes = ["happy", "sad"]

#4. Predict for these classes
tars.predict_zero_shot(sentence, classes)

# Print sentence with predicted labels
print(sentence)
print(sentence.labels[0].value)
print(round(sentence.labels[0].score,2))

Now this code is wrapped into the following function so that I can use it to get predictions for multiple sentences in a dataset.

def tars_zero(example):
  sentence = Sentence(example)
  tars.predict_zero_shot(sentence,classes)
  print(sentence)

inputs = ["I am so glad you liked it!", "I hate it"]
for input in inputs:
  tars_zero(input)

#output: 
Sentence: "I am so glad you liked it !" → happy (0.8667)
Sentence: "I hate it"

Here I'm able to get the prediction only for the first sentence. Can someone tell me how to use TARS Zero Shot Classifier to get predictions for multiple sentences in a dataset? @alanakbik @lukasgarbas @kishaloyhalder @dobbersc @tadejmagajna

[nlproc-007]

Can someone tell me how to use TARS Zero Shot Classifier to get predictions for multiple sentences in a dataset? @alanakbik @lukasgarbas @kishaloyhalder @dobbersc @tadejmagajna

[KaiyeYang-git]

You can try: inputs = ["Sentence("I am so glad you liked it!"), Sentence("I hate it")]

[alanakbik]

Hello @nlproc-007 in this case the problem is that the zero-shot tagger has difficulties applying the label "sad" to the sentence "I hate it". It therefore makes no prediction for this sentence.

If you want to force a prediction for each sentence, you can do this by passing multi_label=False.

Here is an example for your two sentences:

from flair.models import TARSClassifier
from flair.data import Sentence

# 1. Load our pre-trained TARS model for English
tars = TARSClassifier.load('tars-base')

# 2. Define some classes that you want to predict using descriptive names
classes = ["happy", "sad"]

# 3. make a list of sentences
sentences = [
    Sentence("I am so glad you liked it!"),
    Sentence("I hate it"),
]

# 4. predict for both sentences and force single-label prediction with multi_label=False
tars.predict_zero_shot(sentences, classes, multi_label=False)

# 5. print the two sentences with predictions
for sentence in sentences:
    print(sentence)

This should print:

Sentence[8]: "I am so glad you liked it!" → happy (0.8667)
Sentence[3]: "I hate it" → sad (0.0047)

(Note the low probability of 0.0047 on the second sentence. This is because the model struggles to apply the label "sad" to the text "I hate it".)