Test fact checking

[HEmile]

Fact checking basically just converts all predictions into facts and asks the language model whether this fact is true.

Some interesting failure cases

Detection of errors:

To fix:

• None results should be different (ie this has no borders).

[HEmile]

This is not in the dev set as an answer!

Neither is this

[HEmile]

It doesn't work well for the chemical compounds. It'd filter out correct results. But this could maybe be improved with a better prompt.

[HEmile]

PersonLanguage:

Correct

[HEmile]

Occupation:

Incorrect

correct

[HEmile]

PersonInstrument Some good results, but

is incorrect

[HEmile]

Employer:

BUT ALSO

which is incorrect

correct

[HEmile]

Easiest way to implement this is as a separate post processing script on the generated results. Makes it easier to compare

[thiviyanT]

Very interesting indeed! I think we should include this in the paper.