why do you have reverse triples in evaluation?

[apoorvumang]

In code

self.valid_data = self.load_data(data_dir, "valid", reverse=reverse)
self.test_data = self.load_data(data_dir, "test", reverse=reverse)

it should be

self.valid_data = self.load_data(data_dir, "valid", reverse=False)
self.test_data = self.load_data(data_dir, "test", reverse=False)

I did testing with this and the results it shows are much better than reported in the paper. Please let me know if I am wrong.

[apoorvumang]

Much better on FB15k-237, I'm getting MRR 0.40

Haven't tried on other datasets.

These results look like a huge jump so I request you to please run it at your end and verify.

[ibalazevic]

Testing is done in both directions, i.e. we test all the objects with (s, r, ?) and all the subjects with (o, r^-1, ?). Check the paper for more details on this.

When you set reverse=False, you are only testing in one direction, i.e. (s, r, ?), which explains why the results you're getting are better than the ones reported.

[apoorvumang]

thanks!