help, the results of the paper cannot be reproduced.

migalkin / StarE

EMNLP 2020: Message Passing for Hyper-Relational Knowledge Graphs

MIT License

87 stars 17 forks source link

help, the results of the paper cannot be reproduced. #11

Closed zhiweihu1103 closed 2 years ago

zhiweihu1103 commented 2 years ago

Description: I want to reproduce the results of the JF17K dataset in the penultimate row of Table 2, but unfortunately, I used the repository you provided and did not get the results in the paper.

The script: CUDA_VISIBLE_DEVICES=0 python ./run.py MODEL_NAME stare_trans_baseline DATASET jf17k STATEMENT_LEN 3 MAX_QPAIRS 3 SAMPLER_W_QUALIFIERS False

My results: mrr: 0.33328 | mr: 706.71785 | h3: 0.37438 | h5: 0.43126 | h10: 0.51320

Can you help me why this happen?

zhiweihu1103 commented 2 years ago

Sorry, I forgot to set the CLEANED_DATASET False parameter. I will close the issue.