search

peterlipan / ASD_GP_GCN

ASD diagnosis on ABIDE I
MIT License
43 stars 10 forks source link

Accuracy #3

Closed shengaoya closed 2 years ago

shengaoya commented 2 years ago

Hello, I am very interested in your work, but when I reproduce the code. The final result was "GCN 10 fold test set results, loss = 0.777685, accuracy = 0.494253 GCN 10 fold val set results, Loss = 0.720890, accuracy = 0.589744 Mean Accuracy: 0.500562 ", the TRAIN_ACC of MLP is 1. Could you tell me what the problem is? Looking forward to your reply!

peterlipan commented 2 years ago

Hi! Thanks for your attention. I have rechecked and rerun my code and achieved the claimed performance. I need more information to find the possible problem. For example,

  1. Have you implemented all the modules mentioned in our paper/code? e.g., Nested K-fold, modification on Hierarchical graph pooling, population graph construction.
  2. Or, if you have run our code, could you please specify your modification on our settings? If all the parameters are unchanged, could you please provide the number of epochs of training GCN?

Thanks!

shengaoya commented 2 years ago

Hi！ First of all, I didn't change any code，But I downloaded the dataset myself. I downloaded 884 CPAC processed 'Rois_Ho' data and randomly selected 871 of them.And upload it to ./temp/ABIDE_pcp/cpac/filt_global.In the code: the parameters of MLP is: args.times = 3 # repeat times of the second level 10-fold cross-validation args.least = 60 # smallest number of training epochs; avoid under-fitting args.patience = 50 # patience for early stopping args.epochs = 200 # maximum number of epochs args.weight_decay = 0.1 args.nhid = 256

the parameters of GCN is: args.num_features = args.nhid // 2 # output feature size of MLP args.nhid = args.num_features // 2 args.epochs = 100000 # maximum number of training epochs args.patience = 20000 # patience for early stop regarding the performance on val set args.weight_decay = 0.001 args.least = 0 # least number of training epochs In Addition，the 'lr' is 0.0001,'pooling_ratio' is 0.05,and 'dropout_ratio' is 0.01

But the final result was： ... Training GCN on the 9 fold Epoch: 021679 loss_train: 0.215433 acc_train: 0.922096 loss_val: 1.320118 acc_val: 0.576923 time: 348.476660s Optimization Finished! Total time elapsed: 348.476696 GCN 09 fold test set results, loss = 0.853814, accuracy = 0.425287 GCN 09 fold val set results, loss = 0.768147, accuracy = 0.564103 Training GCN on the 10 fold Epoch: 021788 loss_train: 0.201740 acc_train: 0.936261 loss_val: 0.995509 acc_val: 0.576923 time: 346.917461s Optimization Finished! Total time elapsed: 346.917502 GCN 10 fold test set results, loss = 0.913977, accuracy = 0.494253 GCN 10 fold val set results, loss = 0.799268, accuracy = 0.525641

peterlipan commented 2 years ago

Hi! I am not sure if the data difference caused the problem, as some data cleaning exists in download_ABIDE.py. Could you please try the whole pipeline, using the same data, to check it? And, if possible, could you please provide your data, or info about how you downloaded it? I will try to test the model once I am free.

shengaoya commented 2 years ago

Hello, I downloaded it using the script on Github. How should I send the data to you? Could you please also send me your data? I couldn't download it by running the download_ABIDE. Py file. Looking forward to your reply！

shengaoya commented 2 years ago

If possible, my email is 283158330@qq.com

peterlipan commented 2 years ago

I have sent you the data via email. Please find enclosed for detail.

peterlipan commented 2 years ago

Hi! I received an email about this update of issue #3 but cannot find it on GitHub. The HTTPError is caused by the Great Firewall. To solve it, you may need a VPN. If this issue still exists, fill free to leave your email address so that I can send the dataset to you.

Hope you can receive this reply.

------------------ 原始邮件 ------------------ 发件人: "jhonP-Li/ASD_GP_GCN" @.>; 发送时间: 2022年6月29日(星期三) 晚上9:23 @.>; @.>;"State @.>; 主题: Re: [jhonP-Li/ASD_GP_GCN] Accuracy (Issue #3)

Hello! The data I downloaded is also 884 samples! When I run the code, the 'abide_dataset = TUDataset(args.data_dir, name='ABIDE', use_node_attr=True)' in training.py report errors, i.e., urllib.error.HTTPError: HTTP Error 404: Not Found Can I download data directly and put it in a folder and skip this step

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>