search

jiasenlu / vilbert_beta

473 stars 96 forks source link

Model performance on VCR val split #12

Closed JizeCao closed 5 years ago

JizeCao commented 5 years ago

I download the processed VCR data and achieve Q-->A performance about 72.217 with the shared checkpoint, which is slightly lower than the paper claimed. I have used my own processed data and achieve similar performance on both the fine-tuned checkpoint and the pretrained checkpoint after fine tuning on my processed data.

jiasenlu commented 5 years ago

I think 0.1 - 0.2 is due to the randomness. I have trained multiple VCR model for the ensemble, can you try using a different random seed?