Early-stop Decision Attack

[carlini]

With a L2 threshold of 4 (or 10), once we succeed, we can early-stop the decision-only attack. We don't care exactly about finding the best attack -- just if we can find one within the threshold specified, and a lot of work is spent optimizing the attack at the end.

[wielandbrendel]

Here is the author of the Boundary Attack. I second @carlini : if you are fine with an adversarial smaller than the threshold one should definitely use early stopping as most energy is spent on the last 10% rather than the first 90%. This is not yet implemented in Foolbox but it is something we can add if you are interested.

[carlini]

That would be great if you wanted to do this. On defended models, I don't know if I expect this to help that much, because the attack might fail often, but for un-defended models this should speed up evaluation significantly.

[jonasrauber]

early stopping is now supported in all Foolbox attacks including the Boundary Attack: https://github.com/bethgelab/foolbox/pull/213

[jonasrauber]

@carlini I just implemented this and released it as part of Foolbox 1.5.0.

You can use it by changing

self.attack = FoolboxBoundaryAttack(model=Model())

to

mse_threshold = l2_threshold**2 / number_of_pixels
self.attack = FoolboxBoundaryAttack(model=Model(), threshold=mse_threshold)

(for now, the BoundaryAttack threshold for early stopping has to be specified in mean squared error units, assuming images in [0, 1])

[carlini]

Perfect! I'll take a look at this tomorrow. In order to ensure backwards compatibility, I'll probably warn and fall-back to the prior method if the code version is <1.5.

[carlini]

Done in 46347103.