search

tk-rusch / gradientgating

Gradient gating (ICLR 2023)
MIT License
51 stars 4 forks source link

Detailed settings for reproducing the results in paper. #1

Closed yaokl-nju closed 1 year ago

yaokl-nju commented 1 year ago

Hi Rusch,

This is a really exciting work! I tried to reproducing the results (on heterophilic graphs) in paper, but I failed and found that the results are much worse than the results in paper. Can you provide the detailed settings for reproducing the results?

tk-rusch commented 1 year ago

Hi, Thanks for reaching out. You are right, the hyperparameters provided in the heterophilic graphs file are the (rounded) hyperaparameters only for the wisconsin task using G^2-GraphSAGE. Please note that especially texas, cornell and wisconsin are so volatile (every result has a huge standard deviation) that it is not possible to exactly reproduce the result. You can, however, run a small random search including lr, nhid, weight_decay, dropout, G2_exp and nlayers. This should lead to similar results.

I will at some point provide the hyerparameters for film, squirrel and chameleon -- here the standard deviation is much smaller and the results are perfectly reproducible.

Hope that helps.

yaokl-nju commented 1 year ago

Thanks for the reply! I am looking forward to your follow-up release!

AdarshMJ commented 5 months ago

Hi I am trying to reproduce the results for Squirrel and the accuracy I get for G2GCN is really less. I was hoping if the hyperparameters for these datasets are available? Thanks!