DSE-MSU / DeepRobust

A pytorch adversarial library for attack and defense methods on images and graphs
MIT License
957 stars 191 forks source link

Question about PGD graph global attack with CE/CW loss #149

Open yuChen-XD opened 8 months ago

yuChen-XD commented 8 months ago

Hi, I have a doubt about the [test_pgd.py] (https://github.com/DSE-MSU/DeepRobust/blob/756453e894df2acd154c0016b9e25836c8960b27/examples/graph/test_pgd.py#L90-L96). When I use the CE loss, I get basically the same result as you showed in the previous issues, as shown below.

=== testing GCN on clean graph === Test set results: loss= 0.8108 accuracy= 0.8180 === setup attack model === 100%|██████████| 100/100 [01:35<00:00, 1.05it/s] === testing GCN on Evasion attack === Test set results: loss= 1.0158 accuracy= 0.7250 === testing GCN on Poisoning attack === Test set results: loss= 0.9889 accuracy= 0.7320

However, when I replace the loss type in line87 to 'CW', the result, shown below, is not as the same level as the origin paper. I think the CW loss should have a similar ASR as the CE loss.

=== testing GCN on clean graph === Test set results: loss= 0.8108 accuracy= 0.8180 === setup attack model === 100%|██████████| 100/100 [01:28<00:00, 1.13it/s] === testing GCN on Evasion attack === Test set results: loss= 0.9067 accuracy= 0.7750 === testing GCN on Poisoning attack === Test set results: loss= 0.9512 accuracy= 0.7450

Do you get this difference when you test the PGD method? Thanks for your help in advance.