Observation
Results mentioned in the paper (avg across 5 runs)- (NDCG) 0.403±.041 (MRR) 0.439±.078
Results I got (average across 2 runs)- (NDCG) 0.44 (MRR) 0.52
Doubt
Your upper bound (for eg 0.403+0.041 is close to my average 0.44, etc) is my average.
Wanted to know why this difference occurs. If you know something about this, please let me know.
Relevant library versions-
Task- Paper-Label(L2) classification task
Dataset: CS subset
Observation Results mentioned in the paper (avg across 5 runs)- (NDCG) 0.403±.041 (MRR) 0.439±.078 Results I got (average across 2 runs)- (NDCG) 0.44 (MRR) 0.52
Doubt Your upper bound (for eg 0.403+0.041 is close to my average 0.44, etc) is my average.
Wanted to know why this difference occurs. If you know something about this, please let me know.