search

deepakn97 / relationPrediction

ACL 2019: Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs
521 stars 124 forks source link

when evaluation, why remove entities that not seen? #12

Closed koukoulala closed 4 years ago

koukoulala commented 4 years ago

image In your code, I find that when you evaluate the test set, the entities that not appeared in train set would be removed. Can you please explain this operation? And when I reproduce ConvKB by your code, I can get H@10=0.593 in FB15k-237. Why you report 0.471 ?

koukoulala commented 4 years ago

It's just I think if you remove those entities, it would be unfair for other baselines.

chauhanjatin10 commented 4 years ago

Hi @koukoulala We removed the entities which are not a part of the graph since they are not learnt via the graph attention network operations. Any such entities won't be learnt via any method as well. We ran all the other baselines in similar settings, which makes the evaluation fair. The Hits@10 = 0.593, are from our method which initialises our ConvKB decoder with the embeddings learnt from GAT. Simple ConvKB gives around 0.48 . I hope this clarifies your doubts.