Errors in MSRVTT-QA test set

antoyang / FrozenBiLM

[NeurIPS 2022] Zero-Shot Video Question Answering via Frozen Bidirectional Language Models

https://arxiv.org/abs/2206.08155

Apache License 2.0

156 stars 24 forks source link

Errors in MSRVTT-QA test set #10

Closed vateye closed 1 year ago

vateye commented 1 year ago

Hi, I have found some spelling errors in the test set of MSRVTT. For example, "badmitten", "peson", "tenni". How did you handle such ground truth errors during the testing?

antoyang commented 1 year ago

Hi, I do nothing special. I expect these answers to be out of the answer vocabulary hence the model predictions would be incorrect.