search

kuandeng / LightGCN

486 stars 129 forks source link

Inconsistent results of the code with the original paper #3

Open LBJ66 opened 4 years ago

LBJ66 commented 4 years ago

The results in the original paper (LightGCN) 0.1817 0.2518 0.0655 0.1170 0.0416 0.0739 but the result with the code is Best Iter=[45]@[28563.4] recall=[0.18198], precision=[0.05613], ndcg=[0.15523] Maybe the evaluation code provided is wrong?

hexiangnan commented 4 years ago

The paper reports wrong NDCG score (since the previous NDCG implementation has bugs). I will revise the paper later. Your rerun results look good (see the HR score).

On Thu, Apr 30, 2020 at 9:39 AM LBJ66 notifications@github.com wrote:

The results in the original paper (LightGCN) 0.1817 0.2518 0.0655 0.1170 0.0416 0.0739 but the result with the code is Best Iter=[45]@[28563.4] recall=[0.18198], precision=[0.05613], ndcg=[0.15523] Maybe the evaluation code provided is wrong?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kuandeng/LightGCN/issues/3, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI3DDRSD5FZTV3AXWJ6LCDRPDJE3ANCNFSM4MVCNUCQ .

-- Xiangnan HE (Dr) :: Professor :: Vice Dean, School of Data Science :: University of Science and Technology of China (USTC) :: 443 Huangshan Road, Hefei, China 230027 :: (0551) 63607236 (DID) :: hexn@ustc.edu.cn (E) :: http://staff.ustc.edu.cn/~hexn/ (W)

hexiangnan commented 4 years ago

The results in the original paper (LightGCN) 0.1817 0.2518 0.0655 0.1170 0.0416 0.0739 but the result with the code is Best Iter=[45]@[28563.4] recall=[0.18198], precision=[0.05613], ndcg=[0.15523] Maybe the evaluation code provided is wrong?

The paper has been updated with correct NDCG scores (consistent with the github re-run results). You can find the paper in: http://staff.ustc.edu.cn/~hexn/papers/sigir20-LightGCN.pdf