question about evaluation

[bkj]

In the paper, you say

for a given triple, we generate 2*n_e test triples by 1) keeping the subject entity e_s and relation r fixed and replacing the object entity e_o with all possible entities E and by 2) keeping the object entity e_o and relation r fixed and replacing the subject entity e_s with all entities E.

In the evaluate function, it looks like you score all possilbee_o's given an (e_s, e_r) tuple, then compute the rank of the true e_o. So I see how you're doing 1) above, but are you actually doing 2)?

Thanks! ~ Ben

[ibalazevic]

We train and test the model using 1-N scoring, the same way as in Dettmers et al., 2018 (also see here): for each relation, we create an inverse by appending "_reverse" to the relation name. That way we are able to train and test only in one direction (desirable due to 1-N scoring), predicting both head and tail, because our dataset doesn't just contain (e_1, r, e_2), but also (e_2, r_reverse, e_1).

[bkj]

Ah ok -- makes sense I think. Thanks!