search

awslabs / dgl-lifesci

Python package for graph neural networks in chemistry and biology
Apache License 2.0
730 stars 151 forks source link

Codes based on the GCN Predictor in model zoo are not reproducible #212

Open WMX567 opened 1 year ago

WMX567 commented 1 year ago

Hi,

I use the GCN Predictor in this repo

DGL version is 1.0.1 PyTorch version is 1.12.1

I set the random seeds in this way:

random.seed(seed)
np.random.seed(seed)

torch.manual_seed(seed+r)
torch.cuda.manual_seed_all(seed+r)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False

dgl.random.seed(seed+r)

Any idea about how to fix the reproducibility? Thanks!

mufeili commented 1 year ago

DGL-LifeSci just employs a few operators in DGL and PyTorch. If some of these operators have inherent randomness that cannot be addressed by fixing the random seed, then you will still see some randomness in the results you obtain. One way to tackle the issue is to report the mean and std of the performance numbers over multiple random runs. If you are dealing with a small dataset, then running multiple runs will not be infeasible.