Closed Kolt1911 closed 1 year ago
I tried with my local machine without GPU and the results are the following:
>>> archives_adapt[0].summary()
----------
Total inputs: 1296
Average distance: 0.010443303734064102
Total adversarials: 316
Average distance: 0.01583455689251423
Coverage
Original: 0.045490506329113924
Achieved: 0.08511339662447258
----------
Original label: suit
Count: 980
Average distance: 0.008704898878932
----------
Label: stole
Count: 316
Average distance: 0.01583455689251423
----------
I am not sure why Adapt only generates 400 examples. One possible reason is that GPU is not correctly set, and TF cannot use it for acceleration.
For your information, results in MNIST tutorial are written without GPU.
I tried with my local machine without GPU and the results are the following:
>>> archives_adapt[0].summary() ---------- Total inputs: 1296 Average distance: 0.010443303734064102 Total adversarials: 316 Average distance: 0.01583455689251423 Coverage Original: 0.045490506329113924 Achieved: 0.08511339662447258 ---------- Original label: suit Count: 980 Average distance: 0.008704898878932 ---------- Label: stole Count: 316 Average distance: 0.01583455689251423 ----------
I am not sure why Adapt only generates 400 examples. One possible reason is that GPU is not correctly set, and TF cannot use it for acceleration.
For your information, results in MNIST tutorial are written without GPU.
Thanks for your reply!
I checked the status of gpu when running the experiment. It did use the gpu because of high gpu memory usage and power. The env of gpu is probably compatible. Could you give some proposals about what situation the gpu is not correctly set for TF acceleration?
Also, I had tried to set longer fuzzing time from 20min to 60min, but it didn't reach the performance in gpu, either.
I really don't have any idea. How about other GPU-based applications, like training MNIST model from scratch? It takes about 1 second per iteration with colab GPU, and took about 2-3 seconds per iteration with my local machine with RTX 2080.
Try: https://colab.research.google.com/github/kupl/tutorial/test_lenet5.ipynb
I really don't have any idea. How about other GPU-based applications, like training MNIST model from scratch? It takes about 1 second per iteration with colab GPU, and took about 2-3 seconds per iteration with my local machine with RTX 2080.
Try: https://colab.research.google.com/github/kupl/tutorial/test_lenet5.ipynb
Thanks. I'll try to use colab to rerun the experiment.
Thanks. I'll try to use colab to rerun the experiment.
If you find any problem, please let me know.
Thanks. I'll try to use colab to rerun the experiment.
If you find any problem, please let me know.
Hello. I appreciate your help. I have tried to rerun the experiment on the colab with tesla t4 gpu. Unfortunately, the performance is the same as what I had done, that the quantities of adversarial samples is great less than the paper. The following is the output of 10 images. Does anything I could do next?
----------
Total inputs: 13297
Average distance: 0.002051855204626918
Total adversarials: 0
Average distance: -
Coverage
Original: 0.045490506329113924
Achieved: 0.05775316455696203
----------
Original label: Gordon_setter
Count: 13297
Average distance: 0.002051855204626918
----------
----------
Total inputs: 13225
Average distance: 0.010842270217835903
Total adversarials: 889
Average distance: 0.06209832429885864
Coverage
Original: 0.04588607594936709
Achieved: 0.13337289029535865
----------
Original label: dingo
Count: 12336
Average distance: 0.007148476783186197
----------
Label: hyena
Count: 149
Average distance: 0.04001113399863243
----------
Label: Ibizan_hound
Count: 316
Average distance: 0.04688433185219765
----------
Label: basenji
Count: 61
Average distance: 0.06257662177085876
----------
Label: Italian_greyhound
Count: 156
Average distance: 0.07354681938886642
----------
Label: computer_keyboard
Count: 12
Average distance: 0.07971850037574768
----------
Label: shoe_shop
Count: 195
Average distance: 0.09323695302009583
----------
----------
Total inputs: 13108
Average distance: 0.005441570654511452
Total adversarials: 526
Average distance: 0.03273507580161095
Coverage
Original: 0.045490506329113924
Achieved: 0.11438554852320675
----------
Original label: suit
Count: 12582
Average distance: 0.00430054496973753
----------
Label: stole
Count: 526
Average distance: 0.03273507580161095
----------
----------
Total inputs: 13171
Average distance: 0.002221100265160203
Total adversarials: 0
Average distance: -
Coverage
Original: 0.045490506329113924
Achieved: 0.05386339662447257
----------
Original label: freight_car
Count: 13171
Average distance: 0.002221100265160203
----------
----------
Total inputs: 13204
Average distance: 0.006522973533719778
Total adversarials: 112
Average distance: 0.10492376238107681
Coverage
Original: 0.045490506329113924
Achieved: 0.10884757383966245
----------
Original label: matchstick
Count: 13092
Average distance: 0.005681169684976339
----------
Label: thatch
Count: 101
Average distance: 0.10867155343294144
----------
Label: sea_urchin
Count: 11
Average distance: 0.07051229476928711
----------
----------
Total inputs: 13259
Average distance: 0.007664752658456564
Total adversarials: 111
Average distance: 0.04815076291561127
Coverage
Original: 0.045490506329113924
Achieved: 0.11900052742616034
----------
Original label: English_foxhound
Count: 13148
Average distance: 0.007322955876588821
----------
Label: whippet
Count: 5
Average distance: 0.07180314511060715
----------
Label: Saluki
Count: 106
Average distance: 0.047035083174705505
----------
----------
Total inputs: 13020
Average distance: 0.002887626877054572
Total adversarials: 0
Average distance: -
Coverage
Original: 0.04614978902953586
Achieved: 0.07561972573839662
----------
Original label: prayer_rug
Count: 13020
Average distance: 0.002887626877054572
----------
----------
Total inputs: 13037
Average distance: 0.009038560092449188
Total adversarials: 1021
Average distance: 0.050655242055654526
Coverage
Original: 0.04555643459915612
Achieved: 0.1785337552742616
----------
Original label: house_finch
Count: 12016
Average distance: 0.005502388346940279
----------
Label: hen
Count: 35
Average distance: 0.05718062445521355
----------
Label: partridge
Count: 438
Average distance: 0.03958751633763313
----------
Label: rock_python
Count: 39
Average distance: 0.021582739427685738
----------
Label: boa_constrictor
Count: 46
Average distance: 0.022036049515008926
----------
Label: king_snake
Count: 53
Average distance: 0.02394760027527809
----------
Label: quail
Count: 45
Average distance: 0.05112099274992943
----------
Label: puffer
Count: 5
Average distance: 0.04790183901786804
----------
Label: electric_ray
Count: 197
Average distance: 0.06255562603473663
----------
Label: honeycomb
Count: 123
Average distance: 0.09312015026807785
----------
Label: window_screen
Count: 40
Average distance: 0.07341377437114716
----------
----------
Total inputs: 13442
Average distance: 0.0033626507502049208
Total adversarials: 0
Average distance: -
Coverage
Original: 0.04568829113924051
Achieved: 0.0691587552742616
----------
Original label: grasshopper
Count: 13442
Average distance: 0.0033626507502049208
----------
----------
Total inputs: 13335
Average distance: 0.01175614446401596
Total adversarials: 762
Average distance: 0.09106249362230301
Coverage
Original: 0.045490506329113924
Achieved: 0.14425105485232068
----------
Original label: Pomeranian
Count: 12573
Average distance: 0.006949699949473143
----------
Label: cairn
Count: 495
Average distance: 0.06978417187929153
----------
Label: feather_boa
Count: 213
Average distance: 0.12915335595607758
----------
Label: wig
Count: 26
Average distance: 0.11854938417673111
----------
Label: sea_urchin
Count: 28
Average distance: 0.15194664895534515
----------
plt.plot(times, coverages, label='adapt')\ncoverages = []\nfor archive in archives_rand:\n coverage = []\n timestamp = iter(archive.timestamp)\n t, cov = next(timestamp)\n for current_t in times:\n while current_t > t:\n t, cov = next(timestamp)\n coverage.append(cov)\n coverages.append(coverage)\ncoverages = np.mean(coverages, axis=0)\nplt.plot(times, coverages, label='rand')\nplt.suptitle('vgg19')\nplt.legend()\nplt.show()
I think the result with colab is similar to that in the tutorial in terms of the number of generated inputs and coverage (I compared the result of Pomeranian
).
Detailed number could be vary, since Adapt uses some randomness in its algorithm.
Note that the time budget for VGG-19 in the paper is 1 hour for each image.
I think the result with colab is similar to that in the tutorial in terms of the number of generated inputs and coverage (I compared the result of
Pomeranian
). Detailed number could be vary, since Adapt uses some randomness in its algorithm.Note that the time budget for VGG-19 in the paper is 1 hour for each image.
The reason of various number of generated cases is the randomness, but as the result shows, like the "grasshopper", the generated adversarial sample is zero. Is it normal?
The images that Adapt failed in generating an adversarial input (Gordon_setter
, freight_car
, prayer_rug
, grasshopper
) are the ones that Adapt found only a few adversarial examples with 1 hour time budget.
I thiink that VGG-19 is so confident about those images so that Adapt have hard time to fool the model.
Since the performance is similar between the colab and my gpu. I tried 60 min testing per seed on my gpu. But the performance is strange. Some seeds still don't generate adversarial cases, such as 'gordon_setter' and 'freight_car'. It really takes time.
I use Dlfuzz to fuzz the same model using the same seeds, including gordon_setter and freight_car, which are not generate adversarial cases. Maybe the VGG-19 is confident about these images. But some theories say the adversarial samples are dense and widespread. Does these two seeds could generate adversarial samples that I haven't find?
time=3601 vgg19 60min gpu
----------
Total inputs: 30294
Average distance: 0.003002540674060583
Total adversarials: 0
Average distance: -
Coverage
Original: 0.04555643459915612
Achieved: 0.07733386075949367
----------
Original label: Gordon_setter
Count: 30294
Average distance: 0.003002540674060583
----------
----------
Total inputs: 29421
Average distance: 0.0029962738044559956
Total adversarials: 0
Average distance: -
Coverage
Original: 0.04555643459915612
Achieved: 0.05748945147679325
----------
Original label: freight_car
Count: 29421
Average distance: 0.0029962738044559956
----------
----------
Total inputs: 29382
Average distance: 0.0042890808545053005
Total adversarials: 5
Average distance: 0.06863461434841156
Coverage
Original: 0.04562236286919831
Achieved: 0.11122099156118144
----------
Original label: matchstick
Count: 29377
Average distance: 0.004278129432350397
----------
Label: thatch
Count: 5
Average distance: 0.06863461434841156
----------
----------
Total inputs: 29365
Average distance: 0.0074866474606096745
Total adversarials: 1505
Average distance: 0.0637459084391594
Coverage
Original: 0.04647943037974683
Achieved: 0.2066191983122363
----------
Original label: house_finch
Count: 27860
Average distance: 0.004447516519576311
----------
Label: hen
Count: 221
Average distance: 0.0506969690322876
----------
Label: ruffed_grouse
Count: 17
Average distance: 0.05144954100251198
----------
Label: black_swan
Count: 169
Average distance: 0.06703733652830124
----------
Label: goose
Count: 54
Average distance: 0.059602245688438416
----------
Label: brain_coral
Count: 287
Average distance: 0.09754448384046555
----------
Label: partridge
Count: 178
Average distance: 0.042677417397499084
----------
Label: great_grey_owl
Count: 92
Average distance: 0.08370223641395569
----------
Label: starfish
Count: 1
Average distance: 0.09008491039276123
----------
Label: eel
Count: 87
Average distance: 0.03620380535721779
----------
Label: hermit_crab
Count: 15
Average distance: 0.055631332099437714
----------
Label: tick
Count: 127
Average distance: 0.06290798634290695
----------
Label: rock_python
Count: 15
Average distance: 0.06156248226761818
----------
Label: thunder_snake
Count: 13
Average distance: 0.06874621659517288
----------
Label: king_snake
Count: 55
Average distance: 0.07377670705318451
----------
Label: sea_slug
Count: 140
Average distance: 0.04529878869652748
----------
Label: coral_reef
Count: 34
Average distance: 0.05112788826227188
----------
----------
Total inputs: 28550
Average distance: 0.0030834649223834276
Total adversarials: 594
Average distance: 0.031734488904476166
Coverage
Original: 0.045490506329113924
Achieved: 0.10390295358649788
----------
Original label: English_foxhound
Count: 27956
Average distance: 0.0024746970739215612
----------
Label: standard_poodle
Count: 512
Average distance: 0.03266482427716255
----------
Label: English_setter
Count: 82
Average distance: 0.025925572961568832
----------
----------
Total inputs: 26783
Average distance: 0.007209246978163719
Total adversarials: 400
Average distance: 0.09940675646066666
Coverage
Original: 0.045490506329113924
Achieved: 0.14438291139240506
----------
Original label: Pomeranian
Count: 26383
Average distance: 0.005811415147036314
----------
Label: macaque
Count: 26
Average distance: 0.16306938230991364
----------
Label: shower_cap
Count: 330
Average distance: 0.08723078668117523
----------
Label: patas
Count: 44
Average distance: 0.15310774743556976
----------
----------
Total inputs: 23118
Average distance: 0.0033660174813121557
Total adversarials: 314
Average distance: 0.015147182159125805
Coverage
Original: 0.045490506329113924
Achieved: 0.10535337552742616
----------
Original label: suit
Count: 22804
Average distance: 0.0032037964556366205
----------
Label: stole
Count: 307
Average distance: 0.015298482961952686
----------
Label: jean
Count: 7
Average distance: 0.008511511608958244
----------
----------
Total inputs: 24782
Average distance: 0.011327696032822132
Total adversarials: 1296
Average distance: 0.1028788760304451
Coverage
Original: 0.045490506329113924
Achieved: 0.16126054852320676
----------
Original label: dingo
Count: 23486
Average distance: 0.006275736726820469
----------
Label: Ibizan_hound
Count: 399
Average distance: 0.04686149209737778
----------
Label: basenji
Count: 175
Average distance: 0.0776563361287117
----------
Label: shoe_shop
Count: 74
Average distance: 0.09037216752767563
----------
Label: hyena
Count: 140
Average distance: 0.1051555797457695
----------
Label: English_setter
Count: 97
Average distance: 0.11352179199457169
----------
Label: dalmatian
Count: 2
Average distance: 0.11158636957406998
----------
Label: golden_retriever
Count: 51
Average distance: 0.13159075379371643
----------
Label: puffer
Count: 1
Average distance: 0.12310857325792313
----------
Label: jellyfish
Count: 95
Average distance: 0.14943350851535797
----------
Label: pillow
Count: 141
Average distance: 0.1812417060136795
----------
Label: cheetah
Count: 40
Average distance: 0.15034721791744232
----------
Label: jackfruit
Count: 76
Average distance: 0.19320860505104065
----------
Label: peacock
Count: 5
Average distance: 0.2232123613357544
----------
----------
Total inputs: 23571
Average distance: 0.0013311812654137611
Total adversarials: 0
Average distance: -
Coverage
Original: 0.04562236286919831
Achieved: 0.053731540084388185
----------
Original label: grasshopper
Count: 23571
Average distance: 0.0013311812654137611
----------
----------
Total inputs: 22886
Average distance: 0.0026822793297469616
Total adversarials: 0
Average distance: -
Coverage
Original: 0.04588607594936709
Achieved: 0.08333333333333333
----------
Original label: prayer_rug
Count: 22886
Average distance: 0.0026822793297469616
----------
The the detailed results in 2019 as follows (the table only includes the images that failed in your report): | label | original coverage | achieved coverage | total count | adversarial count | adversarial labels |
---|---|---|---|---|---|---|
freight_car |
0.045 | 0.170 | 11860 | 1367 | 14 | |
prayer_rug |
0.045 | 0.220 | 11906 | 1075 | 8 | |
grasshopper |
0.045 | 0.199 | 11847 | 1830 | 11 | |
Gordon_setter |
0.045 | 0.235 | 11905 | 1751 | 22 |
Unfortunately, there is no generated image remaining.
Plus, I tried to run the VGG tutorial with my local environment (with koreaunivpl/adapt
docker image), but I got similar result as you.
When it comes to the randomness, Adapt does have randomness in initializing the neuron selecting strategy and selecting strategy mutation. How many times you had done the experiment of testing vgg19 when you got this 2019 results? What if I repeat the experiment several times? Could I get the result like the 2019?
The experimental results in the paper is an average of results of single run for each image. However, I did experiment multiple times, which are not included in the paper, and I could get consistent results over multiple trials at that time.
I am not sure whether you can get similar result in terms of labels found, but I can get similar result in terms of coverage.
The issue is closed because there is no comments for a month. Feel free to re-open the issue if you have any further questions.
I tried to use test_vgg19.ipynb to run experiment for testing vgg19. But the performance is worse than the examples in the paper, such as "total adversarials" and "total inputs". But I didn't modify any files or arguments in the code of adapt. BTW, the performance of experiment for testing mnist is the same as the examples. I don't know the reason about this problem.
the output of code "archives_adapt[0].summary()" in my testing: ` Total inputs: 10 Average distance: 0.001166449161246419 Total adversarials: 0 Average distance: - Coverage Original: 0.04568829113924051 Achieved: 0.04720464135021097
Original label: Pomeranian Count: 414 Average distance: 0.016496244817972183
Total inputs: 413 Average distance: 0.002656622789800167 Total adversarials: 0 Average distance: - Coverage Original: 0.04562236286919831 Achieved: 0.05195147679324894 `
And the example is:
` Total inputs: 8471 Average distance: 0.02087555266916752 Total adversarials: 1135 Average distance: 0.0999956950545311 Coverage Original: 0.04555643459915612 Achieved: 0.15928270042194093
Original label: Pomeranian Count: 7336 Average distance: 0.008634363301098347
Label: hyena Count: 217 Average distance: 0.05843096598982811
Label: meerkat Count: 255 Average distance: 0.07762718945741653
Label: tick Count: 224 Average distance: 0.10655633360147476
Label: guinea_pig Count: 33 Average distance: 0.09237903356552124
Label: English_setter Count: 44 Average distance: 0.10023358464241028
Label: Petri_dish Count: 6 Average distance: 0.10553994029760361
Label: flatworm Count: 343 Average distance: 0.14118780195713043
Label: shower_cap Count: 13 Average distance: 0.04866204783320427 `