GraphSage doesn't learn.

diningphil / gnn-comparison

Official Repository of "A Fair Comparison of Graph Neural Networks for Graph Classification", ICLR 2020

GNU General Public License v3.0

385 stars 54 forks source link

GraphSage doesn't learn. #50

Closed ssantos97 closed 8 months ago

ssantos97 commented 8 months ago

Hi, I followed all the instructions in the github but turns out that when I'm trying to train the GraphSAGE for Social-1/IMDB-BINARY and Social-DEGREE/IMDB-BINARY the accuracies don't go higher than 0.5 (including training acc) which shows that the predictions are random. The same thing does not happen for GIN where the accuracies increase.

Pytorch 2.0.1 torch-geometric 2.3.1 python 3.10.10

ssantos97 commented 8 months ago

The outputs of the forward for GraphSAGE are all equal for the same iteration, which I think makes sense since the features are all 1.

ssantos97 commented 8 months ago

Also outputs after first layer(x, edge_index) are all equal for the same training iterations.

ssantos97 commented 8 months ago

Same thing occurs for IMDB-MULTI, the accuracies don't go higher than 30%

diningphil commented 8 months ago

Hi, I'm investigating. It's possible it has to do with the newer version of PyG compared to 2020. Have you tried other social/chemical datasets rather than IMDB?

diningphil commented 8 months ago

It is working on my laptop. IMDB-BINARY learns when considering the degree.

Here are the steps to reproduce

source install.sh

Could you please double check that you are executing the commands correctly?

python PrepareDatasets.py DATA/SOCIAL_DEGREE --dataset-name IMDB-BINARY --use-degree --outer-k 10
cp -r DATA/SOCIAL_DEGREE/IMDB-BINARY/ DATA
python Launch_Experiments.py --config-file config_GraphSAGE.yml --dataset-name IMDB-BINARY --result-folder RESULTS --debug

Thanks, Federico

ssantos97 commented 8 months ago

I ran the same exact commands and the error persists :(

ssantos97 commented 8 months ago

Deleted the folder Data and ran the commands you mention and now it seems to work. I don't know what was blocking it from learning but anyways thank you.

diningphil commented 8 months ago

I''m baffled as well.. happy to help! :)

ssantos97 commented 8 months ago

python PrepareDatasets.py DATA/SOCIAL_1 --dataset-name IMDB-BINARY --use-one --outer-k 10 cp -r DATA/SOCIAL_1/IMDB-BINARY/ DATA python Launch_Experiments.py --config-file config_GraphSAGE.yml --dataset-name IMDB-BINARY --result-folder RESULTS --debug

For SOCIAL_1 keeps not working :(

ssantos97 commented 8 months ago

And this happens for all social datasets with use_one

diningphil commented 8 months ago

Thanks for raising the problem. Apparently setting conv.aggr after initialization does not work anymore, so I updated the code and now GraphSAGE learns with "add" aggregation on SOCIAL_1 IMDB-BINARY.

Please pull and retry!

ssantos97 commented 8 months ago

Working now. Thank you.