search

PetarV- / GAT

Graph Attention Networks (https://arxiv.org/abs/1710.10903)
https://petar-v.com/GAT/
MIT License
3.18k stars 642 forks source link

About the accuracy on the Cora dataset. #19

Closed bbjy closed 5 years ago

bbjy commented 5 years ago

Hello, thank you for the great work and releasing your code.

In the README.md you mentioned that "pre_trained/` contains a pre-trained Cora model (achieving 84.4% accuracy on the test set);" ,but the test accuracy of GAT model is 83.0 +- 0.7% as you presented in the paper. Does that mean that using the GAT model doesn't help to improve accuracy? Or is there anything that I haven't understand?

Looking forward to your reply! Thank you!

PetarV- commented 5 years ago

Hello,

Thank you for the kind words and your interest in GAT!

The pre-trained model corresponds to a single training run of GAT, and serves to demonstrate the kind of accuracy that is possible on Cora using GATs.

Conversely, the value reported in the paper is averaged after training 100 times (with different random seeds). So you can think of it as: training and evaluating GAT on Cora (with the given train/test split) will give you ~83% accuracy on average. But if you are lucky, you can achieve accuracies that are slightly higher on individual runs.

Thanks, Petar

bbjy commented 5 years ago

Hello,

Thank you for the kind words and your interest in GAT!

The pre-trained model corresponds to a single training run of GAT, and serves to demonstrate the kind of accuracy that is possible on Cora using GATs.

Conversely, the value reported in the paper is averaged after training 100 times (with different random seeds). So you can think of it as: training and evaluating GAT on Cora (with the given train/test split) will give you ~83% accuracy on average. But if you are lucky, you can achieve accuracies that are slightly higher on individual runs.

Thanks, Petar Oh,I got it. Thank you for your reply!