search

muhanzhang / pytorch_DGCNN

PyTorch implementation of DGCNN
MIT License
372 stars 122 forks source link

Reproduce results in the original paper #15

Closed AmazingSean closed 5 years ago

AmazingSean commented 5 years ago

Hi, I am trying to reproduce the results in the original paper with the code. I run the code with 

./run_DGCNN.sh DD 0

on datasets protein, D&D, Collab and IMDB. I got accuracy around 60 for all of them, which is lower than the results in the paper. Is there any way I can tune the model to reproduce the results?

Thank you

muhanzhang commented 5 years ago

Hi, by changing bsize=1, I ran on the default permutation indices provided, I got MUTAG: 0.8611 PROTEINS: 0.797673 DD: 0.773504 PTC: 0.567647 The gap is within reasonable range. The results of the paper were got from the original torch version by running 10 series of cross validation exps (100 runs per dataset) by the way.

Since all your results are the same, did you simultaneously run multiple datasets? Since I saved all results in one file "acc_results.txt", I guess the program mixed up all the results and return the average of the last 10?

AmazingSean commented 5 years ago

Hi, by changing bsize=1, I ran on the default permutation indices provided, I got MUTAG: 0.8611 PROTEINS: 0.797673 DD: 0.773504 PTC: 0.567647 The gap is within reasonable range. The results of the paper were got from the original torch version by running 10 series of cross validation exps (100 runs per dataset) by the way.

Since all your results are the same, did you simultaneously run multiple datasets? Since I saved all results in one file "acc_results.txt", I guess the program mixed up all the results and return the average of the last 10?

You're right. I didn't notice that all the results are put into acc_results.txt. I rerun them individually and got the results around reported values. Thank you!