search

deepset-ai / FARM

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.
https://farm.deepset.ai
Apache License 2.0
1.73k stars 247 forks source link

Top-n Accuracy for Text Classification Task #801

Closed shivanouri closed 3 years ago

shivanouri commented 3 years ago

Question I would like to calculate the top-3 accuracy of a text classification task, but as it seems, it's just available for question answering task. Generally, I'd like to have the probability of classes for the predictions.

Additional context My dataset has 140 classes. I use TextClassificationProcessor, TextClassificationHead and AdaptiveModel.

I'd appreciate any hints on how to achieve it.

johann-petrak commented 3 years ago

I think when the Inferencer is created or loaded the parameter return_class_probs=True can be used to return all class probabilities, from those you should be able to find out the top 3?

shivanouri commented 3 years ago

Thanks, problem resolved.