search

shijx12 / KQAPro_Baselines

Pytorch implementation of baseline models of KQA Pro, a large-scale dataset of complex question answering over knowledge base.
http://thukeg.gitee.io/kqa-pro/
MIT License
127 stars 22 forks source link

The detail of Bart program model #5

Closed Snnzhao closed 3 years ago

Snnzhao commented 3 years ago

In the train.py, the model is not valid during training. So I copy the valid part from predict.py. But during training at start, the valid acc drop from 0.007 to 0.004 and does not increase during training. I use bart-base as pretrained model. Can you provide more details for the experiment?

Snnzhao commented 3 years ago

I found even if train 25 epochs when the loss drop to 0.003. It outputs a lot program like
"Find(farmer)\Relate(occupation\backward)\Find(George Segal)\And()\>Find(Rutgers University)\< b>QueryRelation()" While the target is "Find(farmer)\Relate(occupationbackward)\Find(George Segal)\And()\Find(Rutgers University)\QueryRelation()"

Snnzhao commented 3 years ago

It sames the split token always wrong

Snnzhao commented 3 years ago

I sames to solve this problem. This problem is caused by the token \ \, whick is not included in pretrained vocab. I solve this problem by replace this two with # and |. Thanks again for such wonderful work.

shijx12 commented 3 years ago

It is due to the error in the latest BART checkpoint. If somebody meets this problem, you can download the previous checkpoint from https://cloud.tsinghua.edu.cn/f/3b59ec6c43034cfc8841/?dl=1 and load it as the pretrained model to achieve our reported results.

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.