Closed ClaudMor closed 3 months ago
Hi ClaudMor,
Thank you for the question! The implementation of the random split of edges is different.
For the original MagNet implementation here: https://github.com/matthew-hirn/magnet/blob/main/src/utils/edge_data.py#L162, we used the split based on the EdgeSplitter
from stellargraph.data
. Also, the random seed for sampling edges for each fold is different.
For the implementation of this package, to make the installation dependency simpler, we implemented the split based on networkx and PyG. And we use only one random seed to generate edges for all train/val folds.
To fully reproduce the results in the MagNet paper, we recommend using the split in the MagNet paper.
Hope this helps!
Hi @XitongSystem,
It definitely helped!
Thank you very much!
Dear authors,
First, thank you very much for this package.
I've tried reproducing MagNet's Link Prediction results on Cora for the Existence sub-task using this package's models and splitting functions, but I keep consistently getting around 78% accuracy on the test set compared to around 82% as reported by the paper.
Below, you may find an MWE that attains 78% accuracy on the test set: I would ask if you could kindly point to me how it could be modified to get the same performance as in the paper.
I wrote the MWE below by applying the following modifications to the Directed Case Study reported in your documentation:
How may I get
0.82
as in the paper?@XitongSystem might also be interested as the first author of MagNet.
Thank you very much, any advice is greatly appreciated!