hop1 results in metaQA

[ToneLi]

Sorry to bother you, I used command you provided in github (metaQA hop1 full-KG), I just get 0.93, while your paper is 0.97, there is a large gap, on the other hand, I found your paper uses ROBERT to obtian the question representation, while in your code, you use BI-LSTM. Can you explain these two questions? expecially one, why I get 0.93 by your parameter? your command: python3 main.py --mode train --relation_dim 200 --hidden_dim 256 \ --gpu 2 --freeze 0 --batch_size 128 --validate_every 5 --hops 2 --lr 0.0005 --entdrop 0.1 --reldrop 0.2 --scoredrop 0.2 \ --decay 1.0 --model ComplEx --patience 5 --ls 0.0 --kg_type half. If this command is not the command of metaQA hop1 full-KG, can you supply the all command about (hop1,hop2, hop3) in full KG metaQA, WebQuestionsSP and all command about (hop1,hop2, hop3) in half KG, at last, the training command about how to get entity embedding in these two KGs, two half KGs. Thanks! I hope your reply can help me!!!

[apoorvumang]

That command is for half kg, for full you need to replace --kg_type half with --kg_type full in the command.

python3 main.py --mode train --relation_dim 200 --hidden_dim 256
--gpu 2 --freeze 0 --batch_size 128 --validate_every 5 --hops 2 --lr 0.0005 --entdrop 0.1 --reldrop 0.2 --scoredrop 0.2
--decay 1.0 --model ComplEx --patience 5 --ls 0.0 --kg_type half

We found that using BiLSTM is good enough when we have a large number of questions as in MetaQA and saves a lot of training time and compute without affecting the results - you should get the same results with the fixed command.

[ToneLi]

Thanks, the commond in hop1, hop2 and hop3 is same? just the parameters "hops" is different? On the other hand, can you supply the half metaQA training command? to get the entity embedding and relation embedding?

[apoorvumang]

Thanks, the commond in hop1, hop2 and hop3 is same? just the parameters "hops" is different

Yes

can you supply the half metaQA training command? to get the entity embedding and relation embedding?

Sure, please create this as another issue so it is easier to track, thanks!