search

acbull / pyHGT

Code for "Heterogeneous Graph Transformer" (WWW'20), which is based on pytorch_geometric
MIT License
775 stars 162 forks source link

Result Inconsistency #47

Open theartpiece opened 2 years ago

theartpiece commented 2 years ago

Relevant library versions-

  1. transformers==4.16.2
  2. PyTorch==1.10.1
  3. Python==3.7.0

Task- Paper-Label(L2) classification task

Dataset: CS subset

Observation Results mentioned in the paper (avg across 5 runs)- (NDCG) 0.403±.041 (MRR) 0.439±.078 Results I got (average across 2 runs)- (NDCG) 0.44 (MRR) 0.52

Doubt Your upper bound (for eg 0.403+0.041 is close to my average 0.44, etc) is my average.

Wanted to know why this difference occurs. If you know something about this, please let me know.