ernie-vil: how to get same val acc using VCR finetuning model

PaddlePaddle / ERNIE

Official implementations for various pre-training models of ERNIE-family, covering topics of Language Understanding & Generation, Multimodal Understanding & Generation, and beyond.

6.32k stars 1.28k forks source link

ernie-vil: how to get same val acc using VCR finetuning model #774

Closed yanan1989 closed 2 years ago

yanan1989 commented 2 years ago

Thanks for your great work. I am using the VCR finetuning model to evaluate validation set. However, I could not get a correct accuracy score as you reported in the paper? I got 40% val accuracy [the paper: 77%] when I used the [ERNIE-ViL-VCR-task-pretrain base] model and preprocessed data by ViLBERT. If you have any advice I should follow up, could you please let me know? Thank you very much!

BinglengTang commented 2 years ago

The [ERNIE-ViL-VCR-task-pretrain base] model is a pre-trained model. You are supposed to fine-tune the model with VCR train-split data before doing the validation.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reopen it. Thank you for your contributions.