Open HEmile opened 2 years ago
This is not in the dev set as an answer!
Neither is this
It doesn't work well for the chemical compounds. It'd filter out correct results. But this could maybe be improved with a better prompt.
PersonLanguage:
Correct
Occupation:
Incorrect
correct
PersonInstrument Some good results, but
is incorrect
Employer:
BUT ALSO
which is incorrect
correct
Easiest way to implement this is as a separate post processing script on the generated results. Makes it easier to compare
Very interesting indeed! I think we should include this in the paper.
Fact checking basically just converts all predictions into facts and asks the language model whether this fact is true.
Some interesting failure cases
Detection of errors:
To fix: