[Bug] The Example of MoNet is not Reproducible

dmlc / dgl

Python package built to ease deep learning on graph, on top of existing DL frameworks.

http://dgl.ai

Apache License 2.0

13.53k stars 3.02k forks source link

[Bug] The Example of MoNet is not Reproducible #6100

Open chwan1016 opened 1 year ago

chwan1016 commented 1 year ago

🐛 Bug

I was running the example of MoNet (https://github.com/dmlc/dgl/tree/master/examples/pytorch/monet). The final accuracy for Cora is typically between 74%~78%. Validation accuracy never exceeds 80%. The expected accuracy is ~81.6%. I'm not sure if this is caused by the upgrade of the backend framework.

Environment

DGL Version (e.g., 1.0): DGL 1.1
Backend Library & Version (e.g., PyTorch 0.4.1, MXNet/Gluon 1.3): PyTorch 2.1.0
OS (e.g., Linux): Linux

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale due to lack of activity. It will be closed if no further activity occurs. Thank you

frozenbugs commented 1 year ago

@chwan1016 thanks for catching it, cora is a pretty small graph so it is very sensitive to small changes, this example is an example that demonstrate the usability of DGL so we did not keep an eye on the acc of the model.

Do you happen to know why the accuracy is dropped? If not, let's change the README to Cora: ~0.75 to avoid confusion in the future.