Trusted-AI / adversarial-robustness-toolbox

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
https://adversarial-robustness-toolbox.readthedocs.io/en/latest/
MIT License
4.88k stars 1.17k forks source link

Deepfool does not work well. #219

Closed hkthirano closed 4 years ago

hkthirano commented 5 years ago

Deepfool does not work well.

Maybe because batch and x_adv[batch_index_1:batch_index_2] are eaual, overshoot parameter is zero.

# Apply overshoot parameter
x_adv[batch_index_1:batch_index_2] = x_adv[batch_index_1:batch_index_2] + \
    (1 + self.epsilon) * (batch - x_adv[batch_index_1:batch_index_2])

If we change it like this, it works well.

# art/attacks/deepfool.py

# Compute perturbation with implicit batching
for batch_id in range(int(np.ceil(x_adv.shape[0] / float(self.batch_size)))):
    batch_index_1, batch_index_2 = batch_id * self.batch_size, (batch_id + 1) * self.batch_size
    # batch = x_adv[batch_index_1:batch_index_2] 
    batch = x_adv[batch_index_1:batch_index_2].copy()
beat-buesser commented 5 years ago

Hi @hkthirano Thank you very much for using ART and asking this question! Could you please provide more information/code about how you define the ART classifier and your model? Does your model predict probabilities or logits?

imTyrant commented 4 years ago

@beat-buesser I agree with @hkthirano that there is a bug. batch is equal to x_adv[batch_index_1:batch_index_2] (line in here ) and operations on batch are in-place, which will cause x_adv2 will always be 0 (line in here).

# art/attacks/evasion/deepfool.py
100 : batch = x_adv[batch_index_1:batch_index_2]
....
142:  batch[active_indices] += r_var[active_indices]
... 
164:  x_adv2 = (1 + self.epsilon) * (batch - x_adv[batch_index_1:batch_index_2])