search

YiLunLee / missing_aware_prompts

Multimodal Prompting with Missing Modalities for Visual Recognition, CVPR'23
https://yilunlee.github.io/missing_aware_prompts/
160 stars 9 forks source link

Error when running code due to missing 'label' key in 'test.jsonl' #10

Closed Eaven21 closed 1 year ago

Eaven21 commented 1 year ago

Hi there,

Thank you for your contribution to this project! I noticed a couple of issues while running the code and I was hoping to get some clarification:

  1. I encountered an error when trying to use the 'hateful_memes' dataset with the vilt.utils.write_hatememes function. Specifically, there is no 'label' key in the 'test.jsonl' file, which causes the function to raise an error. Could you please advise on how to resolve this issue?

  2. I am also unclear about how to handle modalities with missing values in the make_arrow function. Specifically, how should we set the 'img' and 'text' fields in such cases? Should we set 'img' to None and 'text' to an empty string ('')? Any guidance on this would be much appreciated.

YiLunLee commented 1 year ago

Thank you for your question.

  1. I have updated the information in DATA.md, please refer to it for more details.
  2. I handle modalities with missing values in the *_dataset.py, not the make_arrow function. Please refer to *_dataset.py for more details.