search

rakutentech / stAdv

Spatially Transformed Adversarial Examples with TensorFlow
MIT License
72 stars 18 forks source link

Testing in batch doesn't give adversarial examples #8

Closed praveern closed 5 years ago

praveern commented 5 years ago

Hello Berangerd, so I tried testing in batches as per the suggestion provided by you. However the predicted labels for perturbed images don't match the selected random targets. Could you please guide me in how effectively generate spatial adversarial examples in batches?

Thank you!

berangerd commented 5 years ago

In principle you just need to change the first dimension of the input flow (1 in the demo) to match the batch size for your images and associated labels and targets. If it doesn't work please provide your code and the unexpected output you get.

praveern commented 5 years ago

Hello Beranger,

We weren't able to get the code running in batch using the batch-size. We had to go for a very bad approach of looping over the sess.run( ) over GPU. In this case, we experienced that around 20% of the images don't manage to fool the pretrained clean classifier.

What we observed was "Predicted label before perturbation = 7", "Predicted label after perturbation = 7" (example output for the test image no. 0). What do you think would be the reason for such a lower rate of successful attacks?

We changed the model to a stronger architecture, with the intention to perform stronger whitebox transfer attacks.

berangerd commented 5 years ago

Hi,

Regarding performances, please look at https://github.com/rakutentech/stAdv/issues/4, in particular my comment https://github.com/rakutentech/stAdv/issues/4#issuecomment-434706644. The origin of the discrepancy with what is reported in the original paper has not yet been clarified.