search

flyingdoog / PGExplainer

Parameterized Explainer for Graph Neural Network
123 stars 15 forks source link

AUC calculation #1

Closed gui-li closed 3 years ago

gui-li commented 3 years ago

Why the mean auc score calculated in BA-shapes.ipynb is about 99%. I also tried all nodes from three classes in house motif, the auc score is still around 99%. And that performs quite better than the 0.963±0.011 score you presented in the paper. Are the calculations of auc score different?

flyingdoog commented 3 years ago

In this pynb file, hyper parameters are tuned with validation datasets in the inductive setting. In the table, we follow the setting in GNNexplainer and didn’t include the validation. Since it is unfair to carefully tune hyperparameters directly based on the final AUC, we set a reasonable setting and report the AUC. We explain this phenomenon in the appendix C.1

‘Some results are higher than the reported ones in Section 5 because here we adopt validation datasets to tune the hyper-parameters.’