search

pygod-team / pygod

A Python Library for Graph Outlier Detection (Anomaly Detection)
https://pygod.org
BSD 2-Clause "Simplified" License
1.31k stars 127 forks source link

Reproduced performance is different from the report #104

Open trrrrht opened 5 months ago

trrrrht commented 5 months ago

Hi,

I installed the pygod package and tried to reproduce the performance in the paper. However, the results of some models are not as high as the reported ones. For example, if I use the default parameters of Radar to run on the Enron dataset, the AUC is only about 0.6, which is lower than the reported one.

Is this because of the randomness of unsupervised learning models?

Thanks for your attention.

kayzliu commented 3 months ago

I agree it may be because of the randomness of unsupervised learning models. Here is the comments from PyTorch doc:

Completely reproducible results are not guaranteed across PyTorch releases, individual commits, or different platforms.

The subtle changes in the implementation may change the results.