search

facebookresearch / mmf

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)
https://mmf.sh/
Other
5.48k stars 935 forks source link

Error while training on own dataset: ValueError: Criteria used for early stopping (val/hateful_memes/roc_auc) is not present in meter #754

Closed kLabille closed 3 years ago

kLabille commented 3 years ago

❓ Questions and Help

Hi,

I was able to add my own dataset to mmf, however, when I try to train the unimodal image (Image Grid) model on it, I run into an error when evaluating the first epoch:

2021-01-28T11:33:06 | mmf.trainers.core.training_loop: Evaluation time. Running on full validation set...
....
ValueError: Criteria used for early stopping (val/hateful_memes/roc_auc) is not present in meter.

Any help is appreciated. Thank you

HenryJunW commented 3 years ago

Same problem, how do you solve this? Thanks!

kLabille commented 3 years ago

@HenryJunW I solved the problem by changing the "training criteria" value in the config file of the model to train for my new dataset. For instance, to train the Image Grid model on my new dataset called "my_new_dataset", I would need to modify the file projects/my_new_dataset/configs/unimodal/image.yaml and change the training.criteria parameter from hateful_memes/roc_auc to my_new_dataset/roc_auc