How did you set up "real"?

[jrounds]

Oops: Completely wrong title. I found the real image output and moved on to another issue to write this.

Potential bug in the eval loop that might have a minor to negligible impact on results depending on your data set (mostly negligible)

1. attack.py will skip images that are not evaluate to the correct label by the target model. E.g. in my run through it skipped image 854 of my image set.
2. evaluate.py will end up at
   https://github.com/s-huu/TurningWeaknessIntoStrength/blob/master/detect.py#L114-L116

In that scenario the if statement for file existence will evaluate to false, but rather than skip the image that did not exist because attack.py skipped it, that code will silently reuse viewdata and view_data_label from the previous iteration, effectively counting the last image that did exist twice in results.

I am fixing it now, I suspect a simple "else: continue" would suffice but not sure yet.

[s-huu]

Hi. Thanks for raising the issue.

For 1: We only want to attack the images that are classified correctly by the target model. By definition of adversarial example, the perturbed label should be different from its true label rather than the model predicted label for this image. Hence, we only consider images whose true label is the same as the model predicted label. See the definition section in this paper: https://arxiv.org/pdf/1802.00420.pdf

For 2: Thanks for pointing to that. We have updated the code.