TimDettmers
commented
4 years ago This has mostly to do with the data and how it is constructed. Since in many languages are read from left-to-right, including English, the dataset language, most relationships in knowledge graphs are read in a left-to-right manner. For example, you would see and expect (Tom Hanks, won, Academy Award for Best Actor) rather than (Academy Award for Best Actor, won, Tom Hanks). As you can see in this example, left-to-right and subject-verb-object relationships are both the main reasons why triples in English datasets often have a specific natural order.
The second part of the issue, if you reverse it, you still have a valid problem, but it is often much more difficult. It is common, especially in the datasets used in this case, for heads to be subjects or people, and for tails to be objects or things. Since people are unique and things are often not for (h, r, x) you usually predict fewer things and these fewer things are dissimilar to each other, while predicting (x, r, t) is more difficult because a thing is often shared by many people and these people have quite similar other attributes.
For example, if you need to predict (x, won, Academy Award for Best Actor) there are so many possibilities of actors all looking quite similar although only a few of them won the award. (Tom Hanks, won, x) is a much easier problem since their are fewer awards and it is likely that Tom Hanks won the award if he already won many other awards.
Based on my understanding, your left evalution is to (h ,r, ?) predict tail entity and right is (?, r, t) to predict head entity. I understand why you have to use reverse triple. But why the right results are so much worse than the left part? We train the reverse triple as well, and the left/right results should not be differ that much. It does not make sense to me.
My results as example: Mean reciprocal rank left: 0.41416008121148684 Mean reciprocal rank right: 0.20748184567705122 Mean reciprocal rank: 0.310820963444269