microsoft / Oscar

Oscar and VinVL
MIT License
1.04k stars 251 forks source link

Vocabulary of the test split #202

Open gondimjoaom opened 1 year ago

gondimjoaom commented 1 year ago

Hi! Thanks for the written paper and the availabe code.

I have what may be a stupid question, but I didn't find a straight answer to it anywhere:

When evaluating the model with the karpathy test split, some words might not be present on the vocabulary from the train split. What do you do? Simple remove these words from the captions of the test split?