search

cjlyl / FKGC-CIAN

This repo shows the source code of EMNLP 2022 paper: Learning Inter-Entity Interaction for Few-Shot Knowledge Graph Completion.
13 stars 3 forks source link

The reproduction results are very different from the results of the paper. #3

Open guizi597 opened 1 year ago

guizi597 commented 1 year ago

We use "python main.py --fine_tune --lr 8e-5 --few 5 --prefix nelllr8e-5.5shot" to run the program, the final result is: Hits@10=0.454,Hits@5= 0.393,Hits@1=243,MRR=0.319, but the MRR in the paper is 0.383, a difference of 0.064.

cjlyl commented 1 year ago

We re-implemented our model using the command "python main.py --fine_tune --lr 8e-5 --few 5 --prefix nelllr8e-5.5shot" and achieved a result of 0.379/0.530/0.463/0.302, which is comparable to the result reported in our original paper (0.376/0.527/0.453/0.298).

We have saved and uploaded the model parameters and the corresponding log file for the best result. You can use "python main.py --test --prefix nelllr8e-5.5shot_best" to reproduce our experimental results.

guizi597 commented 1 year ago

image Thank you for your suggestion, but I ran the best model you released and still couldn't achieve the performance you mentioned.

cjlyl commented 1 year ago

Perhaps you could use the dataset provided by FAAN, which we have uploaded to this link. https://www.dropbox.com/s/onbhvsg7wie6vej/FAAN_NELL.rar?dl=0 . (Opening this URL may require a VPN.)

huluoboli commented 1 year ago

Can you provide a link to the wiki dataset you used?

Perhaps you could use the dataset provided by FAAN, which we have uploaded to this link. https://www.dropbox.com/s/onbhvsg7wie6vej/FAAN_NELL.rar?dl=0 . (Opening this URL may require a VPN.)