Reproduce Issues - Githubissues

weihua916 / powerful-gnns

How Powerful are Graph Neural Networks?

MIT License

1.16k stars 223 forks source link

Reproduce Issues #13

Open zhuhm1996 opened 4 years ago

zhuhm1996 commented 4 years ago

Hi, I used the same codes and datasets to tune the parameters provided by the paper. The random seed is set by 0. The followings are the results:

Where the first line represents results from the paper and the second line represents experimental results I conducted. As you can see, I can not reproduce the results of the paper on many datasets. Would you tell me how to reproduce your results?

xptree commented 4 years ago

@zhuhm1996 Could you share your hyper-param for row 2, especially for RDT-B and RDT-M5K. We can only achieve 77 in RDT-B and 49 in RDT-M5K.

Thanks!

cruyffturn commented 3 years ago

@zhuhm1996 I think there is a typo in the table. The first five columns are social network datasets and the latter four columns are bioinformatics datasets. In the "EXPERIMENTS" section of the paper it's stated that different hidden units are searched for the two different categories.

weihua916 commented 3 years ago

Hi, apologies for the delayed response. Two points that are often missed are:

Have you correctly specified --graph_pooling_type? We used the sum graph pooling for bio/chemistry datasets and the mean graph pooling for social network datasets.
Have you added --deg_as_tag (use the node degree as the input node feature) for IMDB and COLLAB?

Besides, those datasets are outdated and are too small to rigorously compare different models. I'd suggest working on more interesting and modern graph datasets like https://ogb.stanford.edu .