DeepFool performance issues

[maliwin]

Hello, I've encountered some performance issues related to DeepFool usage.

Here's a minimal example:

import numpy as np

from PIL import Image
from art.classifiers import TensorFlowV2Classifier
from art.attacks.evasion import DeepFool
from tensorflow.keras.applications.resnet_v2 import ResNet50V2, preprocess_input, decode_predictions

import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
formatter = logging.Formatter("[%(levelname)s] %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)

model = ResNet50V2(weights='imagenet')
art_model = TensorFlowV2Classifier(model=model, nb_classes=1000, input_shape=(224, 224, 3), clip_values=(0, 255),  preprocessing=(127.5, 127.5))

target_image = Image.open('test_im1.jpg')
target_image = target_image.resize((224, 224), resample=Image.LANCZOS)
target_image = np.array(target_image, dtype=np.float32)
target_image_arr = np.array([target_image])

print(decode_predictions(art_model.predict(target_image_arr)))  # macaw

attack = DeepFool(classifier=art_model, max_iter=100)

import time
t1 = time.time()
adv = attack.generate(x=target_image_arr)
print(decode_predictions(art_model.predict(adv)))  # still macaw, failed to generate adversarial
t2 = time.time()
print('Time spent %.3f' % (t2 - t1))

test_im1.jpg is the image from the original implementation at https://github.com/LTS4/DeepFool/blob/master/Python/test_im1.jpg.

Not only does the attack fails with the parameters max_iter=100 and the default epsilon=1e-6, but the time spent needed to finish the attack is about 80 seconds on my machine. Changing the classifier activation to None (as per the original author's note to improve numerical stability) doesn't affect the result.

Compared to the original implementation (at https://github.com/LTS4/DeepFool/tree/master/Python ) which is implemented in PyTorch, this is a lot slower. That implementation runs in ~1second (estimated, not measured) and successfully generates an adversarial image in 50 iterations.

In fact, I can't even find parameters which would successfully generate an adversarial example for that image in ART. Changing the epsilon to any value doesn't seem to help.

Some things are different though. The original implementation uses ResNet34 by default, but changing to ResNet50 yields the same results there. Here I do use ResNet50 V2 instead of V1, but the biggest difference is the simplified preprocessing step (preprocessing for V1 involves changing to BGR and dividing by the imagenet mean). Also, I used a tensorflow 2 classifier instead of pytorch like the original implementation (and I didn't test with pytorch). I also don't rescale and crop the image, I simply rescale it to 224x224, but I assume that shouldn't really cause such a drastic difference in the success rate.

OS Windows 10
Python version 3.8.1
ART version 1.3.0
TensorFlow 2.2.0

[beat-buesser]

Hi @maliwin Thank you very much for using ART and always providing such detailed minimal examples!

On a first quick look at the example, could you please try running the attack with the model predicting logits instead of probabilities by adding classifier_activation=None to ResNet50V2? I think DeepFool is much more effective on a model predicting logits.

[maliwin]

Hi @beat-buesser, I actually already did, as mentioned in the issue text 😄 With model = ResNet50V2(weights='imagenet', classifier_activation=None) it unfortunately still does not succeed:

[[('n01818515', 'macaw', 23.64609), ('n01817953', 'African_grey', 12.33581), ('n01819313', 'sulphur-crested_cockatoo', 10.132466), ('n01616318', 'vulture', 8.724613), ('n01820546', 'lorikeet', 7.8928475)]]
DeepFool: 100%|██████████| 1/1 [01:18<00:00, 78.51s/it]
[INFO] Success rate of DeepFool attack: 0.00%
[[('n01818515', 'macaw', 20.871115), ('n01817953', 'African_grey', 10.242853), ('n01819313', 'sulphur-crested_cockatoo', 8.626734), ('n01616318', 'vulture', 8.010866), ('n01829413', 'hornbill', 7.5200796)]]
Time spent 78.689

[beat-buesser]

Oh, I'm sorry, I didn't notice it initially in your first post. Let me try to run your example.

[maliwin]

@beat-buesser I managed to hack together some sort of minimal example for a PyTorch version which seems to (maybe) work:

import numpy as np

from PIL import Image
from art.classifiers import PyTorchClassifier
from art.attacks.evasion import DeepFool

import torch
import torchvision.models as models
import torchvision.transforms as transforms
from torch.autograd import Variable

import logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
formatter = logging.Formatter("[%(levelname)s] %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)

model = models.resnet50(pretrained=True)
model.eval()
art_model = PyTorchClassifier(model, loss=torch.nn.CrossEntropyLoss(),
                              input_shape=(224, 224, 3), nb_classes=1000, clip_values=(0, 255))

target_image = Image.open('test_im1.jpg')
target_image = target_image.resize((224, 224), resample=Image.LANCZOS)

mean = [0.485, 0.456, 0.406]
std = [0.229, 0.224, 0.225]

target_image = np.array(target_image) / 255
target_image = (target_image - mean) / std
target_image = np.moveaxis(target_image, -1, 0)
tt = torch.from_numpy(target_image.astype(np.float32))
im = tt.cuda()

# idx == 88
idx = np.argmax(model.forward(Variable(im[None, :, :, :], requires_grad=True)).data.cpu().numpy().flatten())
attack = DeepFool(classifier=art_model, max_iter=100)
adv = attack.generate(x=im.cpu()[None, ...])
# idx == 134 lorikeet
idx_adv = np.argmax(model.forward(Variable(torch.from_numpy(adv[0][None, ...]).cuda(), requires_grad=True)).data.cpu().numpy().flatten())

Please forgive the messiness, I don't know much about PyTorch and just forced it to work without much thought.

It also runs in a couple of seconds.

I also ran a profiler on both versions, I can attach them here if you think it is necessary. I think the TF version is just much slower than PyTorch due to some implementation details that go above my head currently.

[beat-buesser]

For the TensorFlow v2 example I have noticed that normalising the image from [0, 255] to [0, 1] with target_image_arr = np.array([target_image]) / 255

and defining the classifier as

art_model = TensorFlowV2Classifier(model=model, nb_classes=1000, input_shape=(224, 224, 3), clip_values=(0, 1),
                                   preprocessing=(0.5, 0.5))

lets DeepFool find an adversarial example in 30 iterations.

[maliwin]

@beat-buesser thank you! It seems to work for me that way as well actually, and in about the same time as the PyTorch version.

I would like to also point out (to any possible future readers) that it is both necessary to scale the image and set classifier_activation=None. Scaling itself is not enough.

[beat-buesser]

@maliwin :+1: Thank you very much! I think you have found an unexpected behaviour that we should improve. It looks like the current implementation of DeepFool is expecting images in the range [0, 1]. I think we should either raise an exception in DeepFool if the provided classifier has clip_values other than (0, 1) or maybe better scale the perturbations found by DeepFool for the actual clip_values of classifier.

[maliwin]

@beat-buesser it might be worth mentioning in the docs that it is important to use logits. While I did take note that the original authors pointed out that the softmax layer causes instabilities, I also didn't expect it would completely prevent the algorithm from working.

Thank you again for such a quick response, it really makes the experience of using ART so much better knowing that any issue someone might bump into will be addressed this quickly 😄 It is greatly appreciated.

[hkthirano]

Hi @maliwin. Deepfool does not work well. #219

[beat-buesser]

Hi @hkthirano Thank you very much for linking #219. I had not noticed the last comment in the already closed #219, but we'll investigate and include the solution in the fix for this issue.

[beat-buesser]

Hi @hkthirano and @maliwin I have implemented the solutions discussed here and in #219 in PR #476. Would you be available to provide a review of PR #476 ?