search

malllabiisc / CompGCN

ICLR 2020: Composition-Based Multi-Relational Graph Convolutional Networks
Apache License 2.0
597 stars 107 forks source link

Why performance on head data hugely worse than tail data? #20

Closed renli1024 closed 3 years ago

renli1024 commented 3 years ago

Hi, thanks for such a good job first!

I'm reproducing the model results and notice that the performance on head data (i.e. use tail and inverse relation to predict head) is extremely worse than tail data, whose MRR are 0.2+ and 0.4+ separately. I've tried two configurations, one is using transe for opn and score_func, the other is distmul for opn and score_func, and both exist the above performance gap. Do you have any idea how this phenomenon occur?

Best wishes.

svjan5 commented 3 years ago

Hi @renli1024, This pattern is not specific to our model, you will find it across other methods. Mostly it is because of the nature of the datasets used.