search

thunlp / OpenKE

An Open-Source Package for Knowledge Embedding (KE)
3.83k stars 985 forks source link

Reproducing OpenKE paper results #171

Closed Alekos92 closed 5 years ago

Alekos92 commented 5 years ago

Hello and thank you for your work.

I would like to ask a question regarding the experimental results reported in the paper "OpenKE: An Open Toolkit for Knowledge Embedding", found here: https://www.aclweb.org/anthology/D18-2024

More specifically, I would like to reproduce the link prediction results in Tables 3 and 4. Do these results correspond to the results returned by the framework after testing has concluded? I am having a hard time figuring out if these results are about average accuracy, raw vs filtered, etc.

Kind regards, Alex

THUCSTHanxu13 commented 5 years ago

My experiment in the original paper is based on FB15K and WN18. However, the current experiments are all based on FB15K237 and WN18RR. I have reproduced some models to achieve the state-of-the-art results, and these scripts are released in the folder "./examples/" of the toolkit.

THUCSTHanxu13 commented 5 years ago

The results of the table 3 and 4 are the average accuracy of Hit@10 (filtered).