YiLunLee / missing_aware_prompts

Multimodal Prompting with Missing Modalities for Visual Recognition, CVPR'23
https://yilunlee.github.io/missing_aware_prompts/
175 stars 12 forks source link

inquery about dataset #9

Closed Eaven21 closed 1 year ago

Eaven21 commented 1 year ago

Hi there,

Thank you for your contribution to this project! I noticed a couple of issues while running the code and I was hoping to get some clarification:

  1. I encountered an error when trying to use the 'hateful_memes' dataset with the vilt.utils.write_hatememes function. Specifically, there is no 'label' key in the 'test.jsonl' file, which causes the function to raise an error. Could you please advise on how to resolve this issue?

  2. I am also unclear about how to handle modalities with missing values in the make_arrow function. Specifically, how should we set the 'img' and 'text' fields in such cases? Should we set 'img' to None and 'text' to an empty string ('')? Any guidance on this would be much appreciated.