search

PaddlePaddle / ERNIE

Official implementations for various pre-training models of ERNIE-family, covering topics of Language Understanding & Generation, Multimodal Understanding & Generation, and beyond.
6.32k stars 1.28k forks source link

no --init_pretraining_params argument in ernie-vil #588

Closed yuewan2 closed 4 years ago

yuewan2 commented 4 years ago

I am currently trying to fine-tune ernie-vil and I use the --init_checkpoint argument to continue training my checkpoint. However, the performance does not change much after continuously training the model for 50000 steps. I came across #417 and notice that I should be using --init_pretraining_params instead of --init_checkpoint. However, there does not seem to have --init_pretraining_params implemented in ernie-vil. What should I do?

oyxuan-11 commented 4 years ago

Hi, which script did you used in fine-tune ernie-vil? If you are using run_finetuning.sh script to finetune ernie-vil, the default setting is using --init_pretraining_params in finetune.py 363 lines. So maybe you can try increase your learning rate or adjust your parameter.

yuewan2 commented 4 years ago

checkpoint的问题我大概了解了。我目前用batch_size=8 训练了6个epoch了(learning rate一直为2e-5)。我用 run_inference.sh 对 validation data vcr task qa subtask 进行测试可以达到75%左右的accuracy，我也用run_inference测了train data vcr task qa subtask accuracy可以达到90%以上，但我用 test data vcr task qa subtask 进行测试缺只能达到25%的accuracy。我想请问这种情况的可能原因？

TangDonnie commented 4 years ago

因为这个任务的作者们没有公布测试集合的label，我们提供的是生成预测结果的json文件，测试集合打印的准确率不具备参考意义，详情参见http://visualcommonsense.com/leaderboard/

yuewan2 commented 4 years ago

了解了，感谢！