Closed XapaJIaMnu-at-meta closed 2 months ago
https://github.com/Psycoy/MixEval/blob/03ee6e606d3b5af8fdb2b1da711f5672d0c98482/mix_eval/data/mixeval-2024-06-01/mixeval/free-form.json#L672
The answerset is a bunch of buzzwords from the harry potter books, it should be just Hufflepuff
Hi Nikolay, thanks for proposing this issue.
This is the problem of the original TriviaQA annotation. Such annotation errors have a negligible number, so they won't affect the result significantly.
https://github.com/Psycoy/MixEval/blob/03ee6e606d3b5af8fdb2b1da711f5672d0c98482/mix_eval/data/mixeval-2024-06-01/mixeval/free-form.json#L672
The answerset is a bunch of buzzwords from the harry potter books, it should be just Hufflepuff