malllabiisc / EmbedKGQA

ACL 2020: Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings
Apache License 2.0
415 stars 95 forks source link

Train model on webqsp_full #117

Closed NLPercx closed 2 years ago

NLPercx commented 2 years ago

Do I need to change -- relation_dim from 200 to 128 when training the model with webqSP_full?

apoorvumang commented 2 years ago

Are you training the QA model or KGE model?

NLPercx commented 2 years ago

Thank you for your reply. I train QA models. Using the following command, I got an accuracy of 44 for webqsp_half, which is not quite as accurate as 53.2 in the paper. What might be the problem? Py --mode train -- relation_DIM 200 --do_batch_norm 1 -- GPU 2 --freeze 1 --batch_size 16 -- validATE_every 10 -- Hops webQSP_half -- LR 0.00002 -- entDrop 0.0 --reldrop 0.0 --scoredrop 0.0 -- Decay 1.0 -- Model ComplEx -- Patience 20 --ls 0.05 --l3_reg 0.001 --nb_epochs 200 --outfile half_fbwq) Error (dimension mismatch) using the following command in the websp_full case Py --mode train -- relation_DIM 200 --do_batch_norm 1 -- GPU 2 --freeze 1 --batch_size 16 -- validATE_every 10 -- Hops webQSP_full -- LR 0.00002 -- entDROP 0.0 -- Reldrop 0.0 --scoredrop 0.0 -- Decay 1.0 -- Model ComplEx -- Patience 20 --ls 0.05 --l3_reg 0.001 --nb_epochs 200 --outfile full_fbwq)

NLPercx commented 2 years ago

Thank you for your reply. I train QA models. Using the following command, I got an accuracy of 44 for webqsp_half, which is not quite as accurate as 53.2 in the paper. What might be the problem? python3 main.py --mode train --relation_dim 200 --do_batch_norm 1 --gpu 2 --freeze 1 --batch_size 16 --validate_every 10 --hops webqsp_half --lr 0.00002 --entdrop 0.0 --reldrop 0.0 --scoredrop 0.0 --decay 1.0 --model ComplEx --patience 20 --ls 0.05 --l3_reg 0.001 --nb_epochs 200 --outfile half_fbwq) Error (dimension mismatch) using the following command in the websp_full case python3 main.py --mode train --relation_dim 200 --do_batch_norm 1 --gpu 2 --freeze 1 --batch_size 16 --validate_every 10 --hops webqsp_full --lr 0.00002 --entdrop 0.0 --reldrop 0.0 --scoredrop 0.0 --decay 1.0 --model ComplEx --patience 20 --ls 0.05 --l3_reg 0.001 --nb_epochs 200 --outfile full_fbwq)

apoorvumang commented 2 years ago

Please see the section on relation matching in the paper as well as in this github repo. That will improve on the results. You can also refer to other issues that have been already resolved regarding this matter such as #115

The full embedding has been pretrained with 128-dim as far as I remember, so you will have to either retrain it or change dimension

NLPercx commented 2 years ago

Thanks!