search

ThomasScialom / QuestEval

MIT License
87 stars 26 forks source link

How to set questeval.task=summarization #6

Open tairan-w opened 2 years ago

tairan-w commented 2 years ago

I run the command 

from questeval.questeval_metric import QuestEval
questeval = QuestEval()
questeval.task=summarization

But it fails saying name 'summarization' is not defined. Meanwhile, in the paper QuestEval: Summarization Asks for Fact-based Evaluation, there is no introduction on how the reference is used. So can you give more explanation on that? Thanks!

UntotaufUrlaub commented 1 year ago

Hi,

seems you forgot the "". from questeval.questeval_metric import QuestEval questeval = QuestEval() questeval.task="summarization"

From inspecting the file questeval/questeval_metric.py I think task has to be passed as string.

Additionally it might be more robust to set this in the constructor: from questeval.questeval_metric import QuestEval questeval = QuestEval(task=summarization)