l0 not implemented correctly

[gehuangyi20]

1. function compare(x,y) does not called after following code. https://github.com/carlini/nn_robust_attacks/blob/d2067d5a929c86c6ff5a368f5489c7cbd1e73a8c/l0_attack.py#L161

So, there is no checking whether the attack is success or not.

2. By default self.independent_channels is False Then, we will run following code https://github.com/carlini/nn_robust_attacks/blob/d2067d5a929c86c6ff5a368f5489c7cbd1e73a8c/l0_attack.py#L228 https://github.com/carlini/nn_robust_attacks/blob/d2067d5a929c86c6ff5a368f5489c7cbd1e73a8c/l0_attack.py#L229

So, valid has shape (pixels, channels), totalchange has shape (pixels`)``. Let's consider color image (3 channels). It turns out the shape ofvalidandtotalchange``` not matched.

In the following code, https://github.com/carlini/nn_robust_attacks/blob/d2067d5a929c86c6ff5a368f5489c7cbd1e73a8c/l0_attack.py#L237

You basically change initial channel value (0,0,0) to 0, which is not correct.

[carlini]

I believe this is what I want. If you have valid.shape == (pixels, channels) and you write valid[e] = 0 then numpy will perform the operation valid[e,:] = 0.

Did this code crash for you or somehow not give the solution?

[gehuangyi20]

I confirm that numpy will perform valid[e,:] = 0. Thanks!

Since I have added code to check whether the attack succeeds or not, I do not know whether the original code will crash or not. But, both l2 and li attack have the code to check whether the attack succeeds or not. I do not see any reason why not adding such code in l0 attack. Also, without checking, we do not know whether the found solution works or not.

[carlini]

Yeah, so for some reason I decided in the l0 attack that when it finds a valid solution, to restore the state which is known to be correct.

https://github.com/carlini/nn_robust_attacks/blob/d2067d5a929c86c6ff5a368f5489c7cbd1e73a8c/l0_attack.py#L162-L163

I don't remember why I did it this way, but because I do it, it's not necessary to insert the compare call. (Although putting it in definitely wouldn't be harmful.)

[gehuangyi20]

Thank you for your clarification. I think your assumption is if the loss value is small enough, then it implies the attack will be successful. I add the validation check in my code since I want to do the attack in parallel, and the validation check is a safeguard.