search

zknus / NeurIPS-2023-HANG-Robustness

Adversarial Robustness in Graph Neural Networks: A Hamiltonian Energy Conservation Approach
11 stars 0 forks source link

Issue about the attacked graph #2

Open NozomiSG opened 3 months ago

NozomiSG commented 3 months ago

Thanks for sharing the code. I'd like to ask you some questions about the attacked graph.

When I was trying to reproduce the results of Table 4 in your paper, I realized that the file of the perturbed graph loaded in line 136 of your code run_metattack_rate.py is located locally on your computer. When I tried to test it with citeseer dataset, I found that the results obtained were not as expected, with multiple predictions close to 67%.

Could you provide your perturbated graph files or explain the potential causes of this results? Looking forward to your reply.

NozomiSG commented 3 months ago

I followed your procedures on cora and citeseer datasets and got the following results. Could you please tell me if this result is correct? Looking forward to your reply!

Screenshot 2024-04-01 at 6 04 21 PM
1957787636 commented 2 months ago

I am also meet the same question, have you solved it yet?

1957787636 commented 2 months ago

@zknus If you have time, could you please answer it?

zknus commented 2 months ago

Hi @1957787636 and @NozomiSG ,

Thank you both for your diligence on this project. Apologies for my delayed response; I did not receive a notification from GitHub regarding this issue.

Regarding Table 4, please note that the datasets for Cora and Citeseer were not included. Based on your findings, the HANG model continues to perform robustly, though it does require further fine-tuning to improve its clean accuracy. We have updated and uploaded the run_metattack_rate.py script, which now allows direct loading of the perturbed graph used in Table 4.

For your information, the perturbed graphs were generated using the script available at: https://github.com/DSE-MSU/DeepRobust/blob/master/examples/graph/test_mettack.py

Thank you once again for your contributions and attention to detail.

1957787636 commented 2 months ago

Hi @1957787636 and @NozomiSG ,

Thank you both for your diligence on this project. Apologies for my delayed response; I did not receive a notification from GitHub regarding this issue.

Regarding Table 4, please note that the datasets for Cora and Citeseer were not included. Based on your findings, the HANG model continues to perform robustly, though it does require further fine-tuning to improve its clean accuracy. We have updated and uploaded the run_metattack_rate.py script, which now allows direct loading of the perturbed graph used in Table 4.

For your information, the perturbed graphs were generated using the script available at: https://github.com/DSE-MSU/DeepRobust/blob/master/examples/graph/test_mettack.py

Thank you once again for your contributions and attention to detail.

Thank you for the response! I tried tuning some hyperparameters of HANG, but it didn't improve HANG's performance on less perturbation rate or even clean graphs. The acc of HANG is far below vanilla gcn in these case.