Open YanghaoZYH opened 3 years ago
Hi,
Actually I am also having this question on the evasion performance. In the evasion attack setting of meta-attack, you can see meta-attack does not work on evasion setting (the performance drops only 2% see Table 3 in the paper). I am not sure if it is because the evasion setting in PGD attack paper is a little different from that in Meta-attack. Or maybe you can try the hyper-parameter setting as Kaidi suggested to see if we can get higher attack success rate.
I have also provided my answer in KaidiXu/GCN_ADV_Train#5.
Thanks.
Hi,
I have some updates for this issue. We found that the authors also use the pseudo-labels of test data during attack, which greatly impacts the evasion performance. I have updated the test_pgd.py
to reflect this.
With the new script, we can now get a much lower evasion accuracy.
$ python test_pgd_new.py --dataset cora --seed=0
=== testing GCN on clean graph ===
Test set results: loss= 0.7849 accuracy= 0.8130
=== setup attack model ===
100%|█████████████████████████████████████████████████████████████████████████| 100/100 [00:08<00:00, 11.28it/s]
=== testing GCN on Evasion attack ===
Test set results: loss= 1.0142 accuracy= 0.7340
=== testing GCN on Poisoning attack ===
Test set results: loss= 1.0695 accuracy= 0.7200
Feel free to let me know if you have other questions.
Hi,
I also met the problem described in question 3 (https://github.com/KaidiXu/GCN_ADV_Train/issues/5) When handling the fixed model with PGD graph global attack without retraining, it becomes a white box evasion attack, but the ASR is quite low. I am wondering do you have any clues for dealing with this?
Thanks for your help in advance.