search

facebookresearch / mmf

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)
https://mmf.sh/
Other
5.5k stars 939 forks source link

How to use Visual Bert for Multi Label Classification #404

Closed ravising-h closed 4 years ago

ravising-h commented 4 years ago

❓ Questions and Help

I want to classify memes( I have Images of memes and through Google OCR I have extracted what is written in them) into multiple classes like offensive, mean, abusive .. they can have more than one label at a time. Can some one guide me through mmf configs and repo, how to run this dataset and will this repo be helpful?

apsdehal commented 4 years ago

Hi @ravising-h,

Please have a look at how we run VisualBERT on MMIMDb using logit_bce as a loss: https://github.com/facebookresearch/mmf/blob/master/projects/visual_bert/configs/mmimdb/defaults.yaml. MMIMDb also have multiple labels and we multilabel metrics to measure performance.

To run your custom dataset, just convert it to MMF format of MMIMDb and it should work.

ravising-h commented 4 years ago

Thank You for your Help. I will close the issue once the training starts.

vedanuj commented 4 years ago

@ravising-h Can we close this issue?

ravising-h commented 4 years ago

I am really sorry, I forgot to reply. Thank you for your Help.

mohamadhabash commented 2 years ago

Hi @ravising-h, I have a similar multi-label task of yours and I'm trying to use VisualBERT, can you tell me how you used it? Thanks.

sdevatha96 commented 1 year ago

Hi, Can someone redirect me to a link where visualBERT is explained?