search

THUDM / GraphMAE

GraphMAE: Self-Supervised Masked Graph Autoencoders in KDD'22
468 stars 75 forks source link

Cannot Reproduce Reported Results with Code in PyG Branch #45

Closed RocccYan closed 1 year ago

RocccYan commented 1 year ago

Hi, thanks for releasing the code of your excellent work in self-supervised GNNs!

I tried to reproduce the graph classification results myself. However, after I clone the source code in PyG branch, and I just run the run_graph.sh with use-config as True, the results were lower than the reported ones at 1-2 precent points among different datasets. For example, the accuracy I reproduced on MUTAG was 86.57±1.21, which reported in main branch was 88.19±1.26.

The environment settings of my experiments are: torch-1.8.1+cu101, python-3.7, torch_geometric-2.20;

Could you give some instructions for why this may happen and how can I reproduce the reported performance? Thanks a lot!

THINK2TRY commented 1 year ago

@RocccYan Hi, thanks for your attention to our work! Apologize for the late response. Can you reproduce the results using the DGL implementation? And what's the running environment? As the size of MUTAG is too small, it can result in much variance in the results. And our experiments are conducted on Nvidia 2080Ti. We also find that the results are more unstable in Nvidia 3090.