search

facebookresearch / mmf

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)
https://mmf.sh/
Other
5.5k stars 939 forks source link

Visual BERT Baseline on test data generating 2000 rows #702

Closed Abhiruchi closed 3 years ago

Abhiruchi commented 3 years ago

I was trying to reproduce results from Visual BERT with COCO baseline for hateful memes. However, I see a CSV with 2000 rows is generated which throws an error in the submission page that the expected rows are 1000 only.

Screenshot 2020-11-29 at 12 45 52

Can anyone please look into this and let me know if I am doing anything wrong.

Currently, I am running the following command -

!mmf_predict config=projects/hateful_memes/configs/visual_bert/defaults.yaml dataset=hateful_memes run_type=test checkpoint.resume_zoo=visual_bert.finetuned.hateful_memes.from_coco model=visual_bert

Abhiruchi commented 3 years ago

NVM, got the issue resolved from here by updating the command for phase 1. https://github.com/facebookresearch/mmf/tree/master/projects/hateful_memes#predicting-for-phase-1 Seems like, by default, it predicts with phase 2 data but since the submissions are closed now, this update would be needed in command to predict for phase 1.