In the light of a major bug #18 I recomputed all scores for all datasets that were reported. While I was able to replicate all scores of other datasets within reasonable limits (some scores decreased, and some increased) I was not able to replicate the scores of FB15k. I have the log files of the run that produced the initial results but using the same parameters I am no longer able to receive scores which are close to the reported scores. I am not sure what has gone wrong, but I hypothesize that I used a different version of FB15k where I removed inverse relationships and self-inverse relationships — at the time of these experiments I was not aware that the same issues have been addressed by FB15k-237 (which only removed inverse relationships). I apologize for any confusion and misreporting that this error entailed. The paper and the scores in this repo have been updated.
In the light of a major bug #18 I recomputed all scores for all datasets that were reported. While I was able to replicate all scores of other datasets within reasonable limits (some scores decreased, and some increased) I was not able to replicate the scores of FB15k. I have the log files of the run that produced the initial results but using the same parameters I am no longer able to receive scores which are close to the reported scores. I am not sure what has gone wrong, but I hypothesize that I used a different version of FB15k where I removed inverse relationships and self-inverse relationships — at the time of these experiments I was not aware that the same issues have been addressed by FB15k-237 (which only removed inverse relationships). I apologize for any confusion and misreporting that this error entailed. The paper and the scores in this repo have been updated.
Erroneous reported scores: