search

DSE-MSU / DeepRobust

A pytorch adversarial library for attack and defense methods on images and graphs
MIT License
995 stars 192 forks source link

Questions about the results of PGD adversarial training experiment. #106

Open enderdzz opened 2 years ago

enderdzz commented 2 years ago

I ran the example code for pgd training several times and each time either the model started converging from the beginning or the model never converged (this happened more often). This makes me very confused. Hope to get a reasonable explanation, thanks!

Ubuntu 20.04.2 LTS NVIDIA RTX 3090 Driver Version: 470.42.01 cuda 11.4 pytorch 1.10.2

from deeprobust.image.defense.pgdtraining import PGDtraining
from deeprobust.image.config import defense_params
from deeprobust.image.netmodels.CNN import Net
import torch
from torchvision import datasets, transforms

model = Net()
train_loader = torch.utils.data.DataLoader(
                datasets.MNIST('deeprobust/image/defense/data', train=True, download=True,
                                transform=transforms.Compose([transforms.ToTensor()])),
                                batch_size=100,shuffle=True)

test_loader = torch.utils.data.DataLoader(
              datasets.MNIST('deeprobust/image/defense/data', train=False,
                            transform=transforms.Compose([transforms.ToTensor()])),
                            batch_size=1000,shuffle=True)

defense = PGDtraining(model, 'cuda')
defense.generate(train_loader, test_loader, **defense_params["PGDtraining_MNIST"])

converged result:

$ python adversarial_training.py
Training epoch:  1
Train Epoch: 1 [0/60000 (0%)]   Loss: 2.351372  Accuracy:0.00%
Train Epoch: 1 [2000/60000 (3%)]    Loss: 2.307143  Accuracy:6.00%
Train Epoch: 1 [4000/60000 (7%)]    Loss: 2.305116  Accuracy:8.00%
Train Epoch: 1 [6000/60000 (10%)]   Loss: 1.967147  Accuracy:34.00%
Train Epoch: 1 [8000/60000 (13%)]   Loss: 1.817768  Accuracy:41.00%
Train Epoch: 1 [10000/60000 (17%)]  Loss: 1.465288  Accuracy:40.00%
Train Epoch: 1 [12000/60000 (20%)]  Loss: 1.436128  Accuracy:45.00%
Train Epoch: 1 [14000/60000 (23%)]  Loss: 1.499460  Accuracy:49.00%
Train Epoch: 1 [16000/60000 (27%)]  Loss: 1.219344  Accuracy:50.00%
Train Epoch: 1 [18000/60000 (30%)]  Loss: 1.183390  Accuracy:52.00%
Train Epoch: 1 [20000/60000 (33%)]  Loss: 1.007030  Accuracy:63.00%
Train Epoch: 1 [22000/60000 (37%)]  Loss: 1.360008  Accuracy:50.00%
Train Epoch: 1 [24000/60000 (40%)]  Loss: 1.038515  Accuracy:61.00%
Train Epoch: 1 [26000/60000 (43%)]  Loss: 1.330964  Accuracy:57.00%
Train Epoch: 1 [28000/60000 (47%)]  Loss: 1.194903  Accuracy:56.00%
Train Epoch: 1 [30000/60000 (50%)]  Loss: 0.944261  Accuracy:67.00%
Train Epoch: 1 [32000/60000 (53%)]  Loss: 1.056627  Accuracy:69.00%
Train Epoch: 1 [34000/60000 (57%)]  Loss: 0.901333  Accuracy:61.00%
Train Epoch: 1 [36000/60000 (60%)]  Loss: 0.738910  Accuracy:74.00%
Train Epoch: 1 [38000/60000 (63%)]  Loss: 0.910679  Accuracy:72.00%
Train Epoch: 1 [40000/60000 (67%)]  Loss: 0.731118  Accuracy:75.00%
Train Epoch: 1 [42000/60000 (70%)]  Loss: 0.858622  Accuracy:74.00%
Train Epoch: 1 [44000/60000 (73%)]  Loss: 0.842118  Accuracy:65.00%
Train Epoch: 1 [46000/60000 (77%)]  Loss: 0.589347  Accuracy:79.00%
Train Epoch: 1 [48000/60000 (80%)]  Loss: 0.826281  Accuracy:72.00%
Train Epoch: 1 [50000/60000 (83%)]  Loss: 0.912431  Accuracy:65.00%
Train Epoch: 1 [52000/60000 (87%)]  Loss: 0.614409  Accuracy:74.00%
Train Epoch: 1 [54000/60000 (90%)]  Loss: 0.813190  Accuracy:68.00%
Train Epoch: 1 [56000/60000 (93%)]  Loss: 0.489161  Accuracy:81.00%
Train Epoch: 1 [58000/60000 (97%)]  Loss: 0.845259  Accuracy:70.00%

Test set: Clean loss: 0.111, Clean Accuracy: 9672/10000 (97%)

Test set: Adv loss: 0.638, Adv Accuracy: 7679/10000 (77%)

Training epoch:  2
Train Epoch: 2 [0/60000 (0%)]   Loss: 0.738329  Accuracy:73.00%
Train Epoch: 2 [2000/60000 (3%)]    Loss: 0.589614  Accuracy:76.00%
Train Epoch: 2 [4000/60000 (7%)]    Loss: 0.542444  Accuracy:77.00%
Train Epoch: 2 [6000/60000 (10%)]   Loss: 1.008485  Accuracy:61.00%
Train Epoch: 2 [8000/60000 (13%)]   Loss: 0.520983  Accuracy:79.00%
Train Epoch: 2 [10000/60000 (17%)]  Loss: 0.552439  Accuracy:77.00%
Train Epoch: 2 [12000/60000 (20%)]  Loss: 0.495401  Accuracy:83.00%
Train Epoch: 2 [14000/60000 (23%)]  Loss: 0.600000  Accuracy:77.00%
Train Epoch: 2 [16000/60000 (27%)]  Loss: 0.486602  Accuracy:81.00%
Train Epoch: 2 [18000/60000 (30%)]  Loss: 0.459716  Accuracy:83.00%
Train Epoch: 2 [20000/60000 (33%)]  Loss: 0.543965  Accuracy:78.00%
Train Epoch: 2 [22000/60000 (37%)]  Loss: 0.594635  Accuracy:79.00%
Train Epoch: 2 [24000/60000 (40%)]  Loss: 0.623865  Accuracy:81.00%
Train Epoch: 2 [26000/60000 (43%)]  Loss: 0.605622  Accuracy:81.00%
Train Epoch: 2 [28000/60000 (47%)]  Loss: 0.355015  Accuracy:88.00%
Train Epoch: 2 [30000/60000 (50%)]  Loss: 0.516162  Accuracy:78.00%
Train Epoch: 2 [32000/60000 (53%)]  Loss: 0.758846  Accuracy:73.00%
Train Epoch: 2 [34000/60000 (57%)]  Loss: 0.476912  Accuracy:79.00%
Train Epoch: 2 [36000/60000 (60%)]  Loss: 0.539509  Accuracy:80.00%
Train Epoch: 2 [38000/60000 (63%)]  Loss: 0.563810  Accuracy:77.00%
Train Epoch: 2 [40000/60000 (67%)]  Loss: 0.370645  Accuracy:85.00%
Train Epoch: 2 [42000/60000 (70%)]  Loss: 0.654697  Accuracy:80.00%
Train Epoch: 2 [44000/60000 (73%)]  Loss: 0.525610  Accuracy:79.00%
Train Epoch: 2 [46000/60000 (77%)]  Loss: 0.519859  Accuracy:82.00%
Train Epoch: 2 [48000/60000 (80%)]  Loss: 0.481817  Accuracy:83.00%
Train Epoch: 2 [50000/60000 (83%)]  Loss: 0.726714  Accuracy:72.00%
Train Epoch: 2 [52000/60000 (87%)]  Loss: 0.572366  Accuracy:81.00%
Train Epoch: 2 [54000/60000 (90%)]  Loss: 0.425735  Accuracy:82.00%
Train Epoch: 2 [56000/60000 (93%)]  Loss: 0.470111  Accuracy:80.00%
Train Epoch: 2 [58000/60000 (97%)]  Loss: 0.645734  Accuracy:78.00%

Test set: Clean loss: 0.089, Clean Accuracy: 9735/10000 (97%)

Test set: Adv loss: 0.556, Adv Accuracy: 7998/10000 (80%)

Training epoch:  3
Train Epoch: 3 [0/60000 (0%)]   Loss: 0.585723  Accuracy:72.00%
Train Epoch: 3 [2000/60000 (3%)]    Loss: 0.474457  Accuracy:82.00%
Train Epoch: 3 [4000/60000 (7%)]    Loss: 0.577215  Accuracy:78.00%
Train Epoch: 3 [6000/60000 (10%)]   Loss: 0.606378  Accuracy:79.00%
Train Epoch: 3 [8000/60000 (13%)]   Loss: 0.562200  Accuracy:79.00%
Train Epoch: 3 [10000/60000 (17%)]  Loss: 0.567655  Accuracy:82.00%
Train Epoch: 3 [12000/60000 (20%)]  Loss: 0.622447  Accuracy:73.00%
Train Epoch: 3 [14000/60000 (23%)]  Loss: 0.586644  Accuracy:78.00%
Train Epoch: 3 [16000/60000 (27%)]  Loss: 0.577470  Accuracy:78.00%
Train Epoch: 3 [18000/60000 (30%)]  Loss: 0.615758  Accuracy:80.00%
Train Epoch: 3 [20000/60000 (33%)]  Loss: 0.545503  Accuracy:82.00%
Train Epoch: 3 [22000/60000 (37%)]  Loss: 0.527496  Accuracy:83.00%
Train Epoch: 3 [24000/60000 (40%)]  Loss: 0.600572  Accuracy:80.00%
Train Epoch: 3 [26000/60000 (43%)]  Loss: 0.479553  Accuracy:81.00%
Train Epoch: 3 [28000/60000 (47%)]  Loss: 0.562436  Accuracy:82.00%
Train Epoch: 3 [30000/60000 (50%)]  Loss: 0.630221  Accuracy:74.00%
Train Epoch: 3 [32000/60000 (53%)]  Loss: 0.495753  Accuracy:83.00%
Train Epoch: 3 [34000/60000 (57%)]  Loss: 0.640357  Accuracy:76.00%
Train Epoch: 3 [36000/60000 (60%)]  Loss: 0.703378  Accuracy:73.00%
Train Epoch: 3 [38000/60000 (63%)]  Loss: 0.490503  Accuracy:81.00%
Train Epoch: 3 [40000/60000 (67%)]  Loss: 0.587693  Accuracy:82.00%
Train Epoch: 3 [42000/60000 (70%)]  Loss: 0.906659  Accuracy:78.00%
Train Epoch: 3 [44000/60000 (73%)]  Loss: 0.564295  Accuracy:76.00%
Train Epoch: 3 [46000/60000 (77%)]  Loss: 0.501209  Accuracy:82.00%
Train Epoch: 3 [48000/60000 (80%)]  Loss: 0.763779  Accuracy:74.00%
Train Epoch: 3 [50000/60000 (83%)]  Loss: 0.664224  Accuracy:77.00%
Train Epoch: 3 [52000/60000 (87%)]  Loss: 0.452256  Accuracy:82.00%
Train Epoch: 3 [54000/60000 (90%)]  Loss: 0.694000  Accuracy:71.00%
Train Epoch: 3 [56000/60000 (93%)]  Loss: 0.527738  Accuracy:78.00%
Train Epoch: 3 [58000/60000 (97%)]  Loss: 0.847104  Accuracy:75.00%

Test set: Clean loss: 0.087, Clean Accuracy: 9740/10000 (97%)

Test set: Adv loss: 0.553, Adv Accuracy: 8005/10000 (80%)

Training epoch:  4
Train Epoch: 4 [0/60000 (0%)]   Loss: 0.483007  Accuracy:79.00%
Train Epoch: 4 [2000/60000 (3%)]    Loss: 0.474230  Accuracy:85.00%
Train Epoch: 4 [4000/60000 (7%)]    Loss: 0.822169  Accuracy:78.00%
Train Epoch: 4 [6000/60000 (10%)]   Loss: 0.711972  Accuracy:78.00%
Train Epoch: 4 [8000/60000 (13%)]   Loss: 0.535654  Accuracy:80.00%
Train Epoch: 4 [10000/60000 (17%)]  Loss: 0.547870  Accuracy:79.00%
Train Epoch: 4 [12000/60000 (20%)]  Loss: 0.571576  Accuracy:81.00%
Train Epoch: 4 [14000/60000 (23%)]  Loss: 0.502839  Accuracy:83.00%
Train Epoch: 4 [16000/60000 (27%)]  Loss: 0.478474  Accuracy:82.00%
Train Epoch: 4 [18000/60000 (30%)]  Loss: 0.427998  Accuracy:85.00%
Train Epoch: 4 [20000/60000 (33%)]  Loss: 0.716814  Accuracy:78.00%
Train Epoch: 4 [22000/60000 (37%)]  Loss: 0.435924  Accuracy:82.00%
Train Epoch: 4 [24000/60000 (40%)]  Loss: 0.492980  Accuracy:84.00%
Train Epoch: 4 [26000/60000 (43%)]  Loss: 0.409413  Accuracy:87.00%
Train Epoch: 4 [28000/60000 (47%)]  Loss: 0.614625  Accuracy:82.00%
Train Epoch: 4 [30000/60000 (50%)]  Loss: 0.719382  Accuracy:76.00%
Train Epoch: 4 [32000/60000 (53%)]  Loss: 0.537918  Accuracy:80.00%
Train Epoch: 4 [34000/60000 (57%)]  Loss: 0.456962  Accuracy:83.00%
Train Epoch: 4 [36000/60000 (60%)]  Loss: 0.557531  Accuracy:79.00%
Train Epoch: 4 [38000/60000 (63%)]  Loss: 0.580985  Accuracy:86.00%
Train Epoch: 4 [40000/60000 (67%)]  Loss: 0.446308  Accuracy:88.00%
Train Epoch: 4 [42000/60000 (70%)]  Loss: 0.449158  Accuracy:78.00%
Train Epoch: 4 [44000/60000 (73%)]  Loss: 0.522976  Accuracy:79.00%
Train Epoch: 4 [46000/60000 (77%)]  Loss: 0.692114  Accuracy:78.00%
Train Epoch: 4 [48000/60000 (80%)]  Loss: 0.475491  Accuracy:79.00%
Train Epoch: 4 [50000/60000 (83%)]  Loss: 0.539442  Accuracy:78.00%
Train Epoch: 4 [52000/60000 (87%)]  Loss: 0.477476  Accuracy:83.00%
Train Epoch: 4 [54000/60000 (90%)]  Loss: 0.383621  Accuracy:88.00%
Train Epoch: 4 [56000/60000 (93%)]  Loss: 0.491226  Accuracy:82.00%
Train Epoch: 4 [58000/60000 (97%)]  Loss: 0.444543  Accuracy:84.00%

Test set: Clean loss: 0.087, Clean Accuracy: 9738/10000 (97%)

Test set: Adv loss: 0.552, Adv Accuracy: 7989/10000 (80%)

...

non-converged result:

$ python adversarial_training.py
Training epoch:  1
Train Epoch: 1 [0/60000 (0%)]   Loss: 2.350390  Accuracy:0.00%
Train Epoch: 1 [2000/60000 (3%)]    Loss: 2.307332  Accuracy:6.00%
Train Epoch: 1 [4000/60000 (7%)]    Loss: 2.298372  Accuracy:13.00%
Train Epoch: 1 [6000/60000 (10%)]   Loss: 2.298733  Accuracy:10.00%
Train Epoch: 1 [8000/60000 (13%)]   Loss: 2.321334  Accuracy:5.00%
Train Epoch: 1 [10000/60000 (17%)]  Loss: 2.308038  Accuracy:10.00%
Train Epoch: 1 [12000/60000 (20%)]  Loss: 2.310513  Accuracy:4.00%
Train Epoch: 1 [14000/60000 (23%)]  Loss: 2.299906  Accuracy:12.00%
Train Epoch: 1 [16000/60000 (27%)]  Loss: 2.309903  Accuracy:7.00%
Train Epoch: 1 [18000/60000 (30%)]  Loss: 2.296326  Accuracy:10.00%
Train Epoch: 1 [20000/60000 (33%)]  Loss: 2.304755  Accuracy:12.00%
Train Epoch: 1 [22000/60000 (37%)]  Loss: 2.298422  Accuracy:10.00%
Train Epoch: 1 [24000/60000 (40%)]  Loss: 2.310440  Accuracy:6.00%
Train Epoch: 1 [26000/60000 (43%)]  Loss: 2.298414  Accuracy:14.00%
Train Epoch: 1 [28000/60000 (47%)]  Loss: 2.310302  Accuracy:7.00%
Train Epoch: 1 [30000/60000 (50%)]  Loss: 2.306832  Accuracy:7.00%
Train Epoch: 1 [32000/60000 (53%)]  Loss: 2.299810  Accuracy:18.00%
Train Epoch: 1 [34000/60000 (57%)]  Loss: 2.295929  Accuracy:16.00%
Train Epoch: 1 [36000/60000 (60%)]  Loss: 2.302053  Accuracy:10.00%
Train Epoch: 1 [38000/60000 (63%)]  Loss: 2.308046  Accuracy:9.00%
Train Epoch: 1 [40000/60000 (67%)]  Loss: 2.307829  Accuracy:5.00%
Train Epoch: 1 [42000/60000 (70%)]  Loss: 2.297229  Accuracy:16.00%
Train Epoch: 1 [44000/60000 (73%)]  Loss: 2.299437  Accuracy:12.00%
Train Epoch: 1 [46000/60000 (77%)]  Loss: 2.303272  Accuracy:8.00%
Train Epoch: 1 [48000/60000 (80%)]  Loss: 2.304019  Accuracy:15.00%
Train Epoch: 1 [50000/60000 (83%)]  Loss: 2.303731  Accuracy:8.00%
Train Epoch: 1 [52000/60000 (87%)]  Loss: 2.304092  Accuracy:10.00%
Train Epoch: 1 [54000/60000 (90%)]  Loss: 2.310779  Accuracy:5.00%
Train Epoch: 1 [56000/60000 (93%)]  Loss: 2.304259  Accuracy:10.00%
Train Epoch: 1 [58000/60000 (97%)]  Loss: 2.295508  Accuracy:15.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  2
Train Epoch: 2 [0/60000 (0%)]   Loss: 2.299287  Accuracy:12.00%
Train Epoch: 2 [2000/60000 (3%)]    Loss: 2.297763  Accuracy:11.00%
Train Epoch: 2 [4000/60000 (7%)]    Loss: 2.291243  Accuracy:14.00%
Train Epoch: 2 [6000/60000 (10%)]   Loss: 2.307785  Accuracy:12.00%
Train Epoch: 2 [8000/60000 (13%)]   Loss: 2.290105  Accuracy:17.00%
Train Epoch: 2 [10000/60000 (17%)]  Loss: 2.302024  Accuracy:10.00%
Train Epoch: 2 [12000/60000 (20%)]  Loss: 2.299577  Accuracy:15.00%
Train Epoch: 2 [14000/60000 (23%)]  Loss: 2.308434  Accuracy:7.00%
Train Epoch: 2 [16000/60000 (27%)]  Loss: 2.305592  Accuracy:9.00%
Train Epoch: 2 [18000/60000 (30%)]  Loss: 2.302638  Accuracy:11.00%
Train Epoch: 2 [20000/60000 (33%)]  Loss: 2.288609  Accuracy:15.00%
Train Epoch: 2 [22000/60000 (37%)]  Loss: 2.301723  Accuracy:12.00%
Train Epoch: 2 [24000/60000 (40%)]  Loss: 2.306819  Accuracy:7.00%
Train Epoch: 2 [26000/60000 (43%)]  Loss: 2.302323  Accuracy:10.00%
Train Epoch: 2 [28000/60000 (47%)]  Loss: 2.308265  Accuracy:8.00%
Train Epoch: 2 [30000/60000 (50%)]  Loss: 2.297973  Accuracy:16.00%
Train Epoch: 2 [32000/60000 (53%)]  Loss: 2.299588  Accuracy:9.00%
Train Epoch: 2 [34000/60000 (57%)]  Loss: 2.299993  Accuracy:12.00%
Train Epoch: 2 [36000/60000 (60%)]  Loss: 2.298849  Accuracy:12.00%
Train Epoch: 2 [38000/60000 (63%)]  Loss: 2.314398  Accuracy:5.00%
Train Epoch: 2 [40000/60000 (67%)]  Loss: 2.298567  Accuracy:13.00%
Train Epoch: 2 [42000/60000 (70%)]  Loss: 2.307476  Accuracy:9.00%
Train Epoch: 2 [44000/60000 (73%)]  Loss: 2.299869  Accuracy:10.00%
Train Epoch: 2 [46000/60000 (77%)]  Loss: 2.308184  Accuracy:12.00%
Train Epoch: 2 [48000/60000 (80%)]  Loss: 2.310362  Accuracy:10.00%
Train Epoch: 2 [50000/60000 (83%)]  Loss: 2.302904  Accuracy:12.00%
Train Epoch: 2 [52000/60000 (87%)]  Loss: 2.303359  Accuracy:9.00%
Train Epoch: 2 [54000/60000 (90%)]  Loss: 2.303502  Accuracy:12.00%
Train Epoch: 2 [56000/60000 (93%)]  Loss: 2.304145  Accuracy:9.00%
Train Epoch: 2 [58000/60000 (97%)]  Loss: 2.304217  Accuracy:9.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  3
Train Epoch: 3 [0/60000 (0%)]   Loss: 2.305378  Accuracy:10.00%
Train Epoch: 3 [2000/60000 (3%)]    Loss: 2.302791  Accuracy:11.00%
Train Epoch: 3 [4000/60000 (7%)]    Loss: 2.312362  Accuracy:8.00%
Train Epoch: 3 [6000/60000 (10%)]   Loss: 2.299649  Accuracy:13.00%
Train Epoch: 3 [8000/60000 (13%)]   Loss: 2.313130  Accuracy:4.00%
Train Epoch: 3 [10000/60000 (17%)]  Loss: 2.303658  Accuracy:10.00%
Train Epoch: 3 [12000/60000 (20%)]  Loss: 2.309258  Accuracy:7.00%
Train Epoch: 3 [14000/60000 (23%)]  Loss: 2.302931  Accuracy:10.00%
Train Epoch: 3 [16000/60000 (27%)]  Loss: 2.308639  Accuracy:8.00%
Train Epoch: 3 [18000/60000 (30%)]  Loss: 2.295690  Accuracy:12.00%
Train Epoch: 3 [20000/60000 (33%)]  Loss: 2.298835  Accuracy:13.00%
Train Epoch: 3 [22000/60000 (37%)]  Loss: 2.303637  Accuracy:12.00%
Train Epoch: 3 [24000/60000 (40%)]  Loss: 2.310228  Accuracy:10.00%
Train Epoch: 3 [26000/60000 (43%)]  Loss: 2.297955  Accuracy:12.00%
Train Epoch: 3 [28000/60000 (47%)]  Loss: 2.304008  Accuracy:9.00%
Train Epoch: 3 [30000/60000 (50%)]  Loss: 2.294887  Accuracy:14.00%
Train Epoch: 3 [32000/60000 (53%)]  Loss: 2.295290  Accuracy:13.00%
Train Epoch: 3 [34000/60000 (57%)]  Loss: 2.302445  Accuracy:12.00%
Train Epoch: 3 [36000/60000 (60%)]  Loss: 2.306960  Accuracy:7.00%
Train Epoch: 3 [38000/60000 (63%)]  Loss: 2.295136  Accuracy:16.00%
Train Epoch: 3 [40000/60000 (67%)]  Loss: 2.309524  Accuracy:8.00%
Train Epoch: 3 [42000/60000 (70%)]  Loss: 2.300213  Accuracy:14.00%
Train Epoch: 3 [44000/60000 (73%)]  Loss: 2.305127  Accuracy:6.00%
Train Epoch: 3 [46000/60000 (77%)]  Loss: 2.301436  Accuracy:11.00%
Train Epoch: 3 [48000/60000 (80%)]  Loss: 2.308454  Accuracy:6.00%
Train Epoch: 3 [50000/60000 (83%)]  Loss: 2.305538  Accuracy:6.00%
Train Epoch: 3 [52000/60000 (87%)]  Loss: 2.305963  Accuracy:10.00%
Train Epoch: 3 [54000/60000 (90%)]  Loss: 2.303193  Accuracy:10.00%
Train Epoch: 3 [56000/60000 (93%)]  Loss: 2.298576  Accuracy:11.00%
Train Epoch: 3 [58000/60000 (97%)]  Loss: 2.304191  Accuracy:8.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  4
Train Epoch: 4 [0/60000 (0%)]   Loss: 2.297378  Accuracy:12.00%
Train Epoch: 4 [2000/60000 (3%)]    Loss: 2.291992  Accuracy:15.00%
Train Epoch: 4 [4000/60000 (7%)]    Loss: 2.296839  Accuracy:15.00%
Train Epoch: 4 [6000/60000 (10%)]   Loss: 2.303780  Accuracy:12.00%
Train Epoch: 4 [8000/60000 (13%)]   Loss: 2.302330  Accuracy:8.00%
Train Epoch: 4 [10000/60000 (17%)]  Loss: 2.290765  Accuracy:17.00%
Train Epoch: 4 [12000/60000 (20%)]  Loss: 2.303672  Accuracy:8.00%
Train Epoch: 4 [14000/60000 (23%)]  Loss: 2.302781  Accuracy:12.00%
Train Epoch: 4 [16000/60000 (27%)]  Loss: 2.294488  Accuracy:18.00%
Train Epoch: 4 [18000/60000 (30%)]  Loss: 2.297119  Accuracy:14.00%
Train Epoch: 4 [20000/60000 (33%)]  Loss: 2.300329  Accuracy:12.00%
Train Epoch: 4 [22000/60000 (37%)]  Loss: 2.292231  Accuracy:16.00%
Train Epoch: 4 [24000/60000 (40%)]  Loss: 2.306165  Accuracy:9.00%
Train Epoch: 4 [26000/60000 (43%)]  Loss: 2.302768  Accuracy:10.00%
Train Epoch: 4 [28000/60000 (47%)]  Loss: 2.305586  Accuracy:11.00%
Train Epoch: 4 [30000/60000 (50%)]  Loss: 2.299202  Accuracy:14.00%
Train Epoch: 4 [32000/60000 (53%)]  Loss: 2.291331  Accuracy:19.00%
Train Epoch: 4 [34000/60000 (57%)]  Loss: 2.302965  Accuracy:6.00%
Train Epoch: 4 [36000/60000 (60%)]  Loss: 2.293495  Accuracy:13.00%
Train Epoch: 4 [38000/60000 (63%)]  Loss: 2.291692  Accuracy:21.00%
Train Epoch: 4 [40000/60000 (67%)]  Loss: 2.297336  Accuracy:13.00%
Train Epoch: 4 [42000/60000 (70%)]  Loss: 2.296772  Accuracy:11.00%
Train Epoch: 4 [44000/60000 (73%)]  Loss: 2.299289  Accuracy:11.00%
Train Epoch: 4 [46000/60000 (77%)]  Loss: 2.313699  Accuracy:9.00%
Train Epoch: 4 [48000/60000 (80%)]  Loss: 2.296615  Accuracy:12.00%
Train Epoch: 4 [50000/60000 (83%)]  Loss: 2.308397  Accuracy:4.00%
Train Epoch: 4 [52000/60000 (87%)]  Loss: 2.295847  Accuracy:13.00%
Train Epoch: 4 [54000/60000 (90%)]  Loss: 2.304087  Accuracy:12.00%
Train Epoch: 4 [56000/60000 (93%)]  Loss: 2.306994  Accuracy:9.00%
Train Epoch: 4 [58000/60000 (97%)]  Loss: 2.297593  Accuracy:12.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  5
Train Epoch: 5 [0/60000 (0%)]   Loss: 2.310611  Accuracy:7.00%
Train Epoch: 5 [2000/60000 (3%)]    Loss: 2.307751  Accuracy:8.00%
Train Epoch: 5 [4000/60000 (7%)]    Loss: 2.302980  Accuracy:11.00%
Train Epoch: 5 [6000/60000 (10%)]   Loss: 2.301943  Accuracy:9.00%
Train Epoch: 5 [8000/60000 (13%)]   Loss: 2.305467  Accuracy:7.00%
Train Epoch: 5 [10000/60000 (17%)]  Loss: 2.301400  Accuracy:10.00%
Train Epoch: 5 [12000/60000 (20%)]  Loss: 2.299436  Accuracy:11.00%
Train Epoch: 5 [14000/60000 (23%)]  Loss: 2.289795  Accuracy:16.00%
Train Epoch: 5 [16000/60000 (27%)]  Loss: 2.294433  Accuracy:14.00%
Train Epoch: 5 [18000/60000 (30%)]  Loss: 2.295333  Accuracy:15.00%
Train Epoch: 5 [20000/60000 (33%)]  Loss: 2.311388  Accuracy:7.00%
Train Epoch: 5 [22000/60000 (37%)]  Loss: 2.292236  Accuracy:14.00%
Train Epoch: 5 [24000/60000 (40%)]  Loss: 2.299561  Accuracy:12.00%
Train Epoch: 5 [26000/60000 (43%)]  Loss: 2.301660  Accuracy:8.00%
Train Epoch: 5 [28000/60000 (47%)]  Loss: 2.307293  Accuracy:8.00%
Train Epoch: 5 [30000/60000 (50%)]  Loss: 2.305568  Accuracy:11.00%
Train Epoch: 5 [32000/60000 (53%)]  Loss: 2.300808  Accuracy:12.00%
Train Epoch: 5 [34000/60000 (57%)]  Loss: 2.304881  Accuracy:8.00%
Train Epoch: 5 [36000/60000 (60%)]  Loss: 2.304814  Accuracy:10.00%
Train Epoch: 5 [38000/60000 (63%)]  Loss: 2.294127  Accuracy:11.00%
Train Epoch: 5 [40000/60000 (67%)]  Loss: 2.294360  Accuracy:15.00%
Train Epoch: 5 [42000/60000 (70%)]  Loss: 2.307119  Accuracy:16.00%
Train Epoch: 5 [44000/60000 (73%)]  Loss: 2.309528  Accuracy:6.00%
Train Epoch: 5 [46000/60000 (77%)]  Loss: 2.290560  Accuracy:16.00%
Train Epoch: 5 [48000/60000 (80%)]  Loss: 2.307828  Accuracy:9.00%
Train Epoch: 5 [50000/60000 (83%)]  Loss: 2.301415  Accuracy:12.00%
Train Epoch: 5 [52000/60000 (87%)]  Loss: 2.311077  Accuracy:9.00%
Train Epoch: 5 [54000/60000 (90%)]  Loss: 2.304275  Accuracy:10.00%
Train Epoch: 5 [56000/60000 (93%)]  Loss: 2.296764  Accuracy:11.00%
Train Epoch: 5 [58000/60000 (97%)]  Loss: 2.296467  Accuracy:16.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  6
Train Epoch: 6 [0/60000 (0%)]   Loss: 2.291795  Accuracy:19.00%
Train Epoch: 6 [2000/60000 (3%)]    Loss: 2.303429  Accuracy:8.00%
Train Epoch: 6 [4000/60000 (7%)]    Loss: 2.286690  Accuracy:19.00%
Train Epoch: 6 [6000/60000 (10%)]   Loss: 2.301649  Accuracy:11.00%
Train Epoch: 6 [8000/60000 (13%)]   Loss: 2.299635  Accuracy:12.00%
Train Epoch: 6 [10000/60000 (17%)]  Loss: 2.297937  Accuracy:13.00%
Train Epoch: 6 [12000/60000 (20%)]  Loss: 2.296413  Accuracy:12.00%
Train Epoch: 6 [14000/60000 (23%)]  Loss: 2.309968  Accuracy:8.00%
Train Epoch: 6 [16000/60000 (27%)]  Loss: 2.299039  Accuracy:13.00%
Train Epoch: 6 [18000/60000 (30%)]  Loss: 2.305315  Accuracy:11.00%
Train Epoch: 6 [20000/60000 (33%)]  Loss: 2.302159  Accuracy:10.00%
Train Epoch: 6 [22000/60000 (37%)]  Loss: 2.294314  Accuracy:14.00%
Train Epoch: 6 [24000/60000 (40%)]  Loss: 2.302288  Accuracy:9.00%
Train Epoch: 6 [26000/60000 (43%)]  Loss: 2.298751  Accuracy:13.00%
Train Epoch: 6 [28000/60000 (47%)]  Loss: 2.305173  Accuracy:12.00%
Train Epoch: 6 [30000/60000 (50%)]  Loss: 2.306027  Accuracy:9.00%
Train Epoch: 6 [32000/60000 (53%)]  Loss: 2.299875  Accuracy:12.00%
Train Epoch: 6 [34000/60000 (57%)]  Loss: 2.296871  Accuracy:12.00%
Train Epoch: 6 [36000/60000 (60%)]  Loss: 2.310385  Accuracy:9.00%
Train Epoch: 6 [38000/60000 (63%)]  Loss: 2.297417  Accuracy:11.00%
Train Epoch: 6 [40000/60000 (67%)]  Loss: 2.301563  Accuracy:10.00%
Train Epoch: 6 [42000/60000 (70%)]  Loss: 2.310922  Accuracy:8.00%
Train Epoch: 6 [44000/60000 (73%)]  Loss: 2.307162  Accuracy:9.00%
Train Epoch: 6 [46000/60000 (77%)]  Loss: 2.293105  Accuracy:13.00%
Train Epoch: 6 [48000/60000 (80%)]  Loss: 2.304347  Accuracy:5.00%
Train Epoch: 6 [50000/60000 (83%)]  Loss: 2.300519  Accuracy:10.00%
Train Epoch: 6 [52000/60000 (87%)]  Loss: 2.307288  Accuracy:13.00%
Train Epoch: 6 [54000/60000 (90%)]  Loss: 2.300734  Accuracy:12.00%
Train Epoch: 6 [56000/60000 (93%)]  Loss: 2.300862  Accuracy:9.00%
Train Epoch: 6 [58000/60000 (97%)]  Loss: 2.293636  Accuracy:17.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  7
Train Epoch: 7 [0/60000 (0%)]   Loss: 2.308148  Accuracy:5.00%
Train Epoch: 7 [2000/60000 (3%)]    Loss: 2.302309  Accuracy:13.00%
Train Epoch: 7 [4000/60000 (7%)]    Loss: 2.297184  Accuracy:13.00%
Train Epoch: 7 [6000/60000 (10%)]   Loss: 2.302575  Accuracy:11.00%
Train Epoch: 7 [8000/60000 (13%)]   Loss: 2.293189  Accuracy:16.00%
Train Epoch: 7 [10000/60000 (17%)]  Loss: 2.291222  Accuracy:15.00%
Train Epoch: 7 [12000/60000 (20%)]  Loss: 2.309062  Accuracy:7.00%
Train Epoch: 7 [14000/60000 (23%)]  Loss: 2.306710  Accuracy:12.00%
Train Epoch: 7 [16000/60000 (27%)]  Loss: 2.298207  Accuracy:12.00%
Train Epoch: 7 [18000/60000 (30%)]  Loss: 2.299804  Accuracy:10.00%
Train Epoch: 7 [20000/60000 (33%)]  Loss: 2.299542  Accuracy:11.00%
Train Epoch: 7 [22000/60000 (37%)]  Loss: 2.297464  Accuracy:11.00%
Train Epoch: 7 [24000/60000 (40%)]  Loss: 2.303365  Accuracy:11.00%
Train Epoch: 7 [26000/60000 (43%)]  Loss: 2.299595  Accuracy:10.00%
Train Epoch: 7 [28000/60000 (47%)]  Loss: 2.297976  Accuracy:14.00%
Train Epoch: 7 [30000/60000 (50%)]  Loss: 2.295076  Accuracy:21.00%
Train Epoch: 7 [32000/60000 (53%)]  Loss: 2.308296  Accuracy:6.00%
Train Epoch: 7 [34000/60000 (57%)]  Loss: 2.298097  Accuracy:12.00%
Train Epoch: 7 [36000/60000 (60%)]  Loss: 2.296899  Accuracy:14.00%
Train Epoch: 7 [38000/60000 (63%)]  Loss: 2.303312  Accuracy:11.00%
Train Epoch: 7 [40000/60000 (67%)]  Loss: 2.302184  Accuracy:10.00%
Train Epoch: 7 [42000/60000 (70%)]  Loss: 2.293847  Accuracy:15.00%
Train Epoch: 7 [44000/60000 (73%)]  Loss: 2.302801  Accuracy:10.00%
Train Epoch: 7 [46000/60000 (77%)]  Loss: 2.301077  Accuracy:12.00%
Train Epoch: 7 [48000/60000 (80%)]  Loss: 2.304673  Accuracy:8.00%
Train Epoch: 7 [50000/60000 (83%)]  Loss: 2.292211  Accuracy:13.00%
Train Epoch: 7 [52000/60000 (87%)]  Loss: 2.302819  Accuracy:12.00%
Train Epoch: 7 [54000/60000 (90%)]  Loss: 2.293973  Accuracy:10.00%
Train Epoch: 7 [56000/60000 (93%)]  Loss: 2.304142  Accuracy:12.00%
Train Epoch: 7 [58000/60000 (97%)]  Loss: 2.297058  Accuracy:14.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  8
Train Epoch: 8 [0/60000 (0%)]   Loss: 2.295553  Accuracy:16.00%
Train Epoch: 8 [2000/60000 (3%)]    Loss: 2.299169  Accuracy:10.00%
Train Epoch: 8 [4000/60000 (7%)]    Loss: 2.302304  Accuracy:8.00%
Train Epoch: 8 [6000/60000 (10%)]   Loss: 2.306942  Accuracy:7.00%
Train Epoch: 8 [8000/60000 (13%)]   Loss: 2.301146  Accuracy:13.00%
Train Epoch: 8 [10000/60000 (17%)]  Loss: 2.307351  Accuracy:6.00%
Train Epoch: 8 [12000/60000 (20%)]  Loss: 2.300782  Accuracy:13.00%
Train Epoch: 8 [14000/60000 (23%)]  Loss: 2.307566  Accuracy:8.00%
Train Epoch: 8 [16000/60000 (27%)]  Loss: 2.293267  Accuracy:15.00%
Train Epoch: 8 [18000/60000 (30%)]  Loss: 2.317050  Accuracy:2.00%
Train Epoch: 8 [20000/60000 (33%)]  Loss: 2.306501  Accuracy:6.00%
Train Epoch: 8 [22000/60000 (37%)]  Loss: 2.297255  Accuracy:12.00%
Train Epoch: 8 [24000/60000 (40%)]  Loss: 2.297596  Accuracy:16.00%
Train Epoch: 8 [26000/60000 (43%)]  Loss: 2.301309  Accuracy:10.00%
Train Epoch: 8 [28000/60000 (47%)]  Loss: 2.293631  Accuracy:20.00%
Train Epoch: 8 [30000/60000 (50%)]  Loss: 2.307355  Accuracy:8.00%
Train Epoch: 8 [32000/60000 (53%)]  Loss: 2.301221  Accuracy:13.00%
Train Epoch: 8 [34000/60000 (57%)]  Loss: 2.310025  Accuracy:7.00%
Train Epoch: 8 [36000/60000 (60%)]  Loss: 2.304150  Accuracy:11.00%
Train Epoch: 8 [38000/60000 (63%)]  Loss: 2.301630  Accuracy:10.00%
Train Epoch: 8 [40000/60000 (67%)]  Loss: 2.312716  Accuracy:8.00%
Train Epoch: 8 [42000/60000 (70%)]  Loss: 2.305968  Accuracy:10.00%
Train Epoch: 8 [44000/60000 (73%)]  Loss: 2.303119  Accuracy:6.00%
Train Epoch: 8 [46000/60000 (77%)]  Loss: 2.289108  Accuracy:21.00%
Train Epoch: 8 [48000/60000 (80%)]  Loss: 2.301756  Accuracy:10.00%
Train Epoch: 8 [50000/60000 (83%)]  Loss: 2.299839  Accuracy:13.00%
Train Epoch: 8 [52000/60000 (87%)]  Loss: 2.301025  Accuracy:13.00%
Train Epoch: 8 [54000/60000 (90%)]  Loss: 2.305166  Accuracy:11.00%
Train Epoch: 8 [56000/60000 (93%)]  Loss: 2.307246  Accuracy:9.00%
Train Epoch: 8 [58000/60000 (97%)]  Loss: 2.310022  Accuracy:10.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  9
Train Epoch: 9 [0/60000 (0%)]   Loss: 2.307455  Accuracy:7.00%
Train Epoch: 9 [2000/60000 (3%)]    Loss: 2.297664  Accuracy:10.00%
Train Epoch: 9 [4000/60000 (7%)]    Loss: 2.306723  Accuracy:4.00%
Train Epoch: 9 [6000/60000 (10%)]   Loss: 2.292673  Accuracy:18.00%
Train Epoch: 9 [8000/60000 (13%)]   Loss: 2.302122  Accuracy:10.00%
Train Epoch: 9 [10000/60000 (17%)]  Loss: 2.296191  Accuracy:14.00%
Train Epoch: 9 [12000/60000 (20%)]  Loss: 2.303851  Accuracy:6.00%
Train Epoch: 9 [14000/60000 (23%)]  Loss: 2.290812  Accuracy:15.00%
Train Epoch: 9 [16000/60000 (27%)]  Loss: 2.304184  Accuracy:14.00%
Train Epoch: 9 [18000/60000 (30%)]  Loss: 2.311785  Accuracy:6.00%
Train Epoch: 9 [20000/60000 (33%)]  Loss: 2.308771  Accuracy:7.00%
Train Epoch: 9 [22000/60000 (37%)]  Loss: 2.297172  Accuracy:14.00%
Train Epoch: 9 [24000/60000 (40%)]  Loss: 2.308045  Accuracy:8.00%
Train Epoch: 9 [26000/60000 (43%)]  Loss: 2.296133  Accuracy:14.00%
Train Epoch: 9 [28000/60000 (47%)]  Loss: 2.312340  Accuracy:7.00%
Train Epoch: 9 [30000/60000 (50%)]  Loss: 2.307947  Accuracy:7.00%
Train Epoch: 9 [32000/60000 (53%)]  Loss: 2.296268  Accuracy:13.00%
Train Epoch: 9 [34000/60000 (57%)]  Loss: 2.300317  Accuracy:14.00%
Train Epoch: 9 [36000/60000 (60%)]  Loss: 2.298599  Accuracy:13.00%
Train Epoch: 9 [38000/60000 (63%)]  Loss: 2.302203  Accuracy:11.00%
Train Epoch: 9 [40000/60000 (67%)]  Loss: 2.302318  Accuracy:10.00%
Train Epoch: 9 [42000/60000 (70%)]  Loss: 2.301330  Accuracy:10.00%
Train Epoch: 9 [44000/60000 (73%)]  Loss: 2.296648  Accuracy:14.00%
Train Epoch: 9 [46000/60000 (77%)]  Loss: 2.300428  Accuracy:11.00%
Train Epoch: 9 [48000/60000 (80%)]  Loss: 2.300591  Accuracy:8.00%
Train Epoch: 9 [50000/60000 (83%)]  Loss: 2.302067  Accuracy:7.00%
Train Epoch: 9 [52000/60000 (87%)]  Loss: 2.296873  Accuracy:13.00%
Train Epoch: 9 [54000/60000 (90%)]  Loss: 2.298505  Accuracy:12.00%
Train Epoch: 9 [56000/60000 (93%)]  Loss: 2.297913  Accuracy:14.00%
Train Epoch: 9 [58000/60000 (97%)]  Loss: 2.289938  Accuracy:16.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  10
Train Epoch: 10 [0/60000 (0%)]  Loss: 2.292661  Accuracy:15.00%
Train Epoch: 10 [2000/60000 (3%)]   Loss: 2.301641  Accuracy:12.00%
Train Epoch: 10 [4000/60000 (7%)]   Loss: 2.299823  Accuracy:10.00%
Train Epoch: 10 [6000/60000 (10%)]  Loss: 2.311009  Accuracy:7.00%
Train Epoch: 10 [8000/60000 (13%)]  Loss: 2.291386  Accuracy:18.00%
Train Epoch: 10 [10000/60000 (17%)] Loss: 2.297846  Accuracy:14.00%
Train Epoch: 10 [12000/60000 (20%)] Loss: 2.297466  Accuracy:15.00%
Train Epoch: 10 [14000/60000 (23%)] Loss: 2.302794  Accuracy:9.00%
Train Epoch: 10 [16000/60000 (27%)] Loss: 2.303216  Accuracy:10.00%
Train Epoch: 10 [18000/60000 (30%)] Loss: 2.307558  Accuracy:8.00%
Train Epoch: 10 [20000/60000 (33%)] Loss: 2.300370  Accuracy:7.00%
Train Epoch: 10 [22000/60000 (37%)] Loss: 2.298914  Accuracy:15.00%
Train Epoch: 10 [24000/60000 (40%)] Loss: 2.304092  Accuracy:9.00%
Train Epoch: 10 [26000/60000 (43%)] Loss: 2.306258  Accuracy:10.00%
Train Epoch: 10 [28000/60000 (47%)] Loss: 2.292514  Accuracy:14.00%
Train Epoch: 10 [30000/60000 (50%)] Loss: 2.295465  Accuracy:12.00%
Train Epoch: 10 [32000/60000 (53%)] Loss: 2.303174  Accuracy:7.00%
Train Epoch: 10 [34000/60000 (57%)] Loss: 2.313246  Accuracy:5.00%
Train Epoch: 10 [36000/60000 (60%)] Loss: 2.299214  Accuracy:12.00%
Train Epoch: 10 [38000/60000 (63%)] Loss: 2.294871  Accuracy:14.00%
Train Epoch: 10 [40000/60000 (67%)] Loss: 2.299199  Accuracy:13.00%
Train Epoch: 10 [42000/60000 (70%)] Loss: 2.296280  Accuracy:14.00%
Train Epoch: 10 [44000/60000 (73%)] Loss: 2.304071  Accuracy:8.00%
Train Epoch: 10 [46000/60000 (77%)] Loss: 2.295803  Accuracy:14.00%
Train Epoch: 10 [48000/60000 (80%)] Loss: 2.297951  Accuracy:13.00%
Train Epoch: 10 [50000/60000 (83%)] Loss: 2.304263  Accuracy:9.00%
Train Epoch: 10 [52000/60000 (87%)] Loss: 2.302366  Accuracy:11.00%
Train Epoch: 10 [54000/60000 (90%)] Loss: 2.299824  Accuracy:13.00%
Train Epoch: 10 [56000/60000 (93%)] Loss: 2.293972  Accuracy:13.00%
Train Epoch: 10 [58000/60000 (97%)] Loss: 2.302086  Accuracy:9.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

model saved in ./defense_model/
Training epoch:  11
Train Epoch: 11 [0/60000 (0%)]  Loss: 2.294476  Accuracy:14.00%
Train Epoch: 11 [2000/60000 (3%)]   Loss: 2.300943  Accuracy:10.00%
Train Epoch: 11 [4000/60000 (7%)]   Loss: 2.302277  Accuracy:8.00%
Train Epoch: 11 [6000/60000 (10%)]  Loss: 2.290444  Accuracy:17.00%
Train Epoch: 11 [8000/60000 (13%)]  Loss: 2.298254  Accuracy:13.00%
Train Epoch: 11 [10000/60000 (17%)] Loss: 2.303739  Accuracy:8.00%
Train Epoch: 11 [12000/60000 (20%)] Loss: 2.299954  Accuracy:12.00%
Train Epoch: 11 [14000/60000 (23%)] Loss: 2.304588  Accuracy:9.00%
Train Epoch: 11 [16000/60000 (27%)] Loss: 2.308084  Accuracy:7.00%
Train Epoch: 11 [18000/60000 (30%)] Loss: 2.302630  Accuracy:9.00%
Train Epoch: 11 [20000/60000 (33%)] Loss: 2.303596  Accuracy:10.00%
Train Epoch: 11 [22000/60000 (37%)] Loss: 2.298480  Accuracy:12.00%
Train Epoch: 11 [24000/60000 (40%)] Loss: 2.298232  Accuracy:13.00%
Train Epoch: 11 [26000/60000 (43%)] Loss: 2.302874  Accuracy:9.00%
Train Epoch: 11 [28000/60000 (47%)] Loss: 2.299083  Accuracy:13.00%
Train Epoch: 11 [30000/60000 (50%)] Loss: 2.304332  Accuracy:7.00%
Train Epoch: 11 [32000/60000 (53%)] Loss: 2.297570  Accuracy:12.00%
Train Epoch: 11 [34000/60000 (57%)] Loss: 2.302894  Accuracy:9.00%
Train Epoch: 11 [36000/60000 (60%)] Loss: 2.313110  Accuracy:8.00%
Train Epoch: 11 [38000/60000 (63%)] Loss: 2.304519  Accuracy:8.00%
Train Epoch: 11 [40000/60000 (67%)] Loss: 2.301382  Accuracy:8.00%
Train Epoch: 11 [42000/60000 (70%)] Loss: 2.301649  Accuracy:10.00%
Train Epoch: 11 [44000/60000 (73%)] Loss: 2.304024  Accuracy:9.00%
Train Epoch: 11 [46000/60000 (77%)] Loss: 2.306634  Accuracy:9.00%
Train Epoch: 11 [48000/60000 (80%)] Loss: 2.302698  Accuracy:11.00%
Train Epoch: 11 [50000/60000 (83%)] Loss: 2.313467  Accuracy:9.00%
Train Epoch: 11 [52000/60000 (87%)] Loss: 2.288162  Accuracy:17.00%
Train Epoch: 11 [54000/60000 (90%)] Loss: 2.291266  Accuracy:15.00%
Train Epoch: 11 [56000/60000 (93%)] Loss: 2.302409  Accuracy:11.00%
Train Epoch: 11 [58000/60000 (97%)] Loss: 2.295777  Accuracy:17.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  12
Train Epoch: 12 [0/60000 (0%)]  Loss: 2.301293  Accuracy:10.00%
Train Epoch: 12 [2000/60000 (3%)]   Loss: 2.305774  Accuracy:8.00%
Train Epoch: 12 [4000/60000 (7%)]   Loss: 2.289127  Accuracy:16.00%
Train Epoch: 12 [6000/60000 (10%)]  Loss: 2.304395  Accuracy:9.00%
Train Epoch: 12 [8000/60000 (13%)]  Loss: 2.305515  Accuracy:9.00%
Train Epoch: 12 [10000/60000 (17%)] Loss: 2.310131  Accuracy:8.00%
Train Epoch: 12 [12000/60000 (20%)] Loss: 2.309848  Accuracy:5.00%
Train Epoch: 12 [14000/60000 (23%)] Loss: 2.300098  Accuracy:12.00%
Train Epoch: 12 [16000/60000 (27%)] Loss: 2.295067  Accuracy:15.00%
Train Epoch: 12 [18000/60000 (30%)] Loss: 2.305720  Accuracy:10.00%
Train Epoch: 12 [20000/60000 (33%)] Loss: 2.304822  Accuracy:10.00%
Train Epoch: 12 [22000/60000 (37%)] Loss: 2.299730  Accuracy:15.00%
Train Epoch: 12 [24000/60000 (40%)] Loss: 2.299071  Accuracy:12.00%
Train Epoch: 12 [26000/60000 (43%)] Loss: 2.311258  Accuracy:6.00%
Train Epoch: 12 [28000/60000 (47%)] Loss: 2.306416  Accuracy:11.00%
Train Epoch: 12 [30000/60000 (50%)] Loss: 2.292176  Accuracy:15.00%
Train Epoch: 12 [32000/60000 (53%)] Loss: 2.293614  Accuracy:15.00%
Train Epoch: 12 [34000/60000 (57%)] Loss: 2.298029  Accuracy:14.00%
Train Epoch: 12 [36000/60000 (60%)] Loss: 2.301921  Accuracy:12.00%
Train Epoch: 12 [38000/60000 (63%)] Loss: 2.300542  Accuracy:10.00%
Train Epoch: 12 [40000/60000 (67%)] Loss: 2.298026  Accuracy:9.00%
Train Epoch: 12 [42000/60000 (70%)] Loss: 2.310563  Accuracy:3.00%
Train Epoch: 12 [44000/60000 (73%)] Loss: 2.308033  Accuracy:7.00%
Train Epoch: 12 [46000/60000 (77%)] Loss: 2.307054  Accuracy:7.00%
Train Epoch: 12 [48000/60000 (80%)] Loss: 2.298719  Accuracy:11.00%
Train Epoch: 12 [50000/60000 (83%)] Loss: 2.308835  Accuracy:8.00%
Train Epoch: 12 [52000/60000 (87%)] Loss: 2.299920  Accuracy:15.00%
Train Epoch: 12 [54000/60000 (90%)] Loss: 2.296973  Accuracy:13.00%
Train Epoch: 12 [56000/60000 (93%)] Loss: 2.309264  Accuracy:7.00%
Train Epoch: 12 [58000/60000 (97%)] Loss: 2.304202  Accuracy:13.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  13
Train Epoch: 13 [0/60000 (0%)]  Loss: 2.296383  Accuracy:15.00%
Train Epoch: 13 [2000/60000 (3%)]   Loss: 2.295229  Accuracy:13.00%
Train Epoch: 13 [4000/60000 (7%)]   Loss: 2.305760  Accuracy:9.00%
Train Epoch: 13 [6000/60000 (10%)]  Loss: 2.301375  Accuracy:10.00%
Train Epoch: 13 [8000/60000 (13%)]  Loss: 2.301755  Accuracy:10.00%
Train Epoch: 13 [10000/60000 (17%)] Loss: 2.298226  Accuracy:16.00%
Train Epoch: 13 [12000/60000 (20%)] Loss: 2.300665  Accuracy:9.00%
Train Epoch: 13 [14000/60000 (23%)] Loss: 2.293666  Accuracy:11.00%
Train Epoch: 13 [16000/60000 (27%)] Loss: 2.299053  Accuracy:9.00%
Train Epoch: 13 [18000/60000 (30%)] Loss: 2.313861  Accuracy:6.00%
Train Epoch: 13 [20000/60000 (33%)] Loss: 2.303191  Accuracy:7.00%
Train Epoch: 13 [22000/60000 (37%)] Loss: 2.301455  Accuracy:11.00%
Train Epoch: 13 [24000/60000 (40%)] Loss: 2.296962  Accuracy:12.00%
Train Epoch: 13 [26000/60000 (43%)] Loss: 2.307686  Accuracy:7.00%
Train Epoch: 13 [28000/60000 (47%)] Loss: 2.304018  Accuracy:15.00%
Train Epoch: 13 [30000/60000 (50%)] Loss: 2.304168  Accuracy:8.00%
Train Epoch: 13 [32000/60000 (53%)] Loss: 2.292088  Accuracy:17.00%
Train Epoch: 13 [34000/60000 (57%)] Loss: 2.307365  Accuracy:7.00%
Train Epoch: 13 [36000/60000 (60%)] Loss: 2.293342  Accuracy:12.00%
Train Epoch: 13 [38000/60000 (63%)] Loss: 2.311764  Accuracy:8.00%
Train Epoch: 13 [40000/60000 (67%)] Loss: 2.303753  Accuracy:17.00%
Train Epoch: 13 [42000/60000 (70%)] Loss: 2.305759  Accuracy:9.00%
Train Epoch: 13 [44000/60000 (73%)] Loss: 2.302584  Accuracy:14.00%
Train Epoch: 13 [46000/60000 (77%)] Loss: 2.306628  Accuracy:5.00%
Train Epoch: 13 [48000/60000 (80%)] Loss: 2.302365  Accuracy:12.00%
Train Epoch: 13 [50000/60000 (83%)] Loss: 2.300722  Accuracy:14.00%
Train Epoch: 13 [52000/60000 (87%)] Loss: 2.299921  Accuracy:11.00%
Train Epoch: 13 [54000/60000 (90%)] Loss: 2.298523  Accuracy:10.00%
Train Epoch: 13 [56000/60000 (93%)] Loss: 2.295936  Accuracy:15.00%
Train Epoch: 13 [58000/60000 (97%)] Loss: 2.305124  Accuracy:9.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

Training epoch:  14
Train Epoch: 14 [0/60000 (0%)]  Loss: 2.299936  Accuracy:12.00%
Train Epoch: 14 [2000/60000 (3%)]   Loss: 2.298723  Accuracy:13.00%
Train Epoch: 14 [4000/60000 (7%)]   Loss: 2.307172  Accuracy:6.00%
Train Epoch: 14 [6000/60000 (10%)]  Loss: 2.298621  Accuracy:15.00%
Train Epoch: 14 [8000/60000 (13%)]  Loss: 2.292523  Accuracy:15.00%
Train Epoch: 14 [10000/60000 (17%)] Loss: 2.309271  Accuracy:11.00%
Train Epoch: 14 [12000/60000 (20%)] Loss: 2.304428  Accuracy:8.00%
Train Epoch: 14 [14000/60000 (23%)] Loss: 2.288184  Accuracy:19.00%
Train Epoch: 14 [16000/60000 (27%)] Loss: 2.299775  Accuracy:13.00%
Train Epoch: 14 [18000/60000 (30%)] Loss: 2.297093  Accuracy:19.00%
Train Epoch: 14 [20000/60000 (33%)] Loss: 2.291045  Accuracy:16.00%
Train Epoch: 14 [22000/60000 (37%)] Loss: 2.295631  Accuracy:15.00%
Train Epoch: 14 [24000/60000 (40%)] Loss: 2.302824  Accuracy:12.00%
Train Epoch: 14 [26000/60000 (43%)] Loss: 2.299780  Accuracy:12.00%
Train Epoch: 14 [28000/60000 (47%)] Loss: 2.299515  Accuracy:9.00%
Train Epoch: 14 [30000/60000 (50%)] Loss: 2.302814  Accuracy:7.00%
Train Epoch: 14 [32000/60000 (53%)] Loss: 2.300488  Accuracy:12.00%
Train Epoch: 14 [34000/60000 (57%)] Loss: 2.299926  Accuracy:14.00%
Train Epoch: 14 [36000/60000 (60%)] Loss: 2.308850  Accuracy:11.00%
Train Epoch: 14 [38000/60000 (63%)] Loss: 2.295351  Accuracy:16.00%
Train Epoch: 14 [40000/60000 (67%)] Loss: 2.300726  Accuracy:11.00%
Train Epoch: 14 [42000/60000 (70%)] Loss: 2.305223  Accuracy:9.00%
Train Epoch: 14 [44000/60000 (73%)] Loss: 2.304767  Accuracy:11.00%
Train Epoch: 14 [46000/60000 (77%)] Loss: 2.290427  Accuracy:16.00%
Train Epoch: 14 [48000/60000 (80%)] Loss: 2.295404  Accuracy:14.00%
Train Epoch: 14 [50000/60000 (83%)] Loss: 2.297324  Accuracy:13.00%
Train Epoch: 14 [52000/60000 (87%)] Loss: 2.299049  Accuracy:12.00%
Train Epoch: 14 [54000/60000 (90%)] Loss: 2.299668  Accuracy:11.00%
Train Epoch: 14 [56000/60000 (93%)] Loss: 2.301958  Accuracy:14.00%
Train Epoch: 14 [58000/60000 (97%)] Loss: 2.313469  Accuracy:8.00%

Test set: Clean loss: 2.301, Clean Accuracy: 1135/10000 (11%)

Test set: Adv loss: 2.301, Adv Accuracy: 1135/10000 (11%)

...
lizituo0926-bit commented 2 years ago

请问您找到原因了吗