Error on PixelAttack and ThresholdAttack

[akshayag]

File "/media/dellserver/HD1/adversarial-robustness-toolbox-master2/art/attacks/evasion/pixel_threshold.py", line 210, in _attack from cma import CMAOptions ModuleNotFoundError: No module named 'cma'

[beat-buesser]

Hi @akshayag Thank you very much for using ART! cma is one of the dependencies specific to pixel_threshold.py. You can fix this error by installing cma with pip install cma.

[akshayag]

Hello, Thank you so much for your quick response. I have installed the dependency and attack ran for a couple of hours. However, later gives the following error:

Traceback (most recent call last): File "Adversarial_Robustness2.py", line 130, in _train_adv = adv_crafter.generate(x_train) File "/media/dellserver/HD1/adversarial-robustness-toolbox-master2/art/attacks/attack.py", line 70, in replacement_function return fdict[func_name](self, *args, *kwargs) File "/media/dellserver/HD1/adversarial-robustness-toolbox-master2/art/attacks/evasion/pixel_threshold.py", line 175, in generate "Success rate of Attack: %.2f%%", 100 compute_success(self.classifier, x, y, adv_x_best, self.targeted, 1)

File "/media/dellserver/HD1/adversarial-robustness-toolbox-master2/art/utils.py", line 307, in compute_success adv_preds = np.argmax(classifier.predict(x_adv, batch_size=batch_size), axis=1) File "/media/dellserver/HD1/adversarial-robustness-toolbox-master2/art/classifiers/classifier.py", line 67, in replacement_function return fdict[func_name](self, *args, **kwargs)

File "/media/dellserver/HD1/adversarial-robustness-toolbox-master2/art/classifiers/keras.py", line 370, in predict xpreprocessed, = self._apply_preprocessing(x, y=None, fit=False) File "/media/dellserver/HD1/adversarial-robustness-toolbox-master2/art/classifiers/classifier.py", line 231, in _apply_preprocessing x_preprocessed = self._apply_preprocessing_standardisation(x_preprocessed) File "/media/dellserver/HD1/adversarial-robustness-toolbox-master2/art/classifiers/classifier.py", line 282, in _apply_preprocessing_standardisation res = x - sub TypeError: unsupported operand type(s) for -: 'list' and 'int'

Implementation details: CIFAR-10 database (error on the training set) baseline CNN model provided in the example code of the toolbox

[beat-buesser]

@akshayag Could you please share a python script or notebook which reproduces your observation?

[akshayag]

from future import absolute_import, division, print_function, unicode_literals

import logging

from keras.models import Sequential from keras.layers import Dense, Flatten, Conv2D, MaxPooling2D, Activation, Dropout import numpy as np

from art.attacks import DeepFool from art.attacks import PixelAttack from art.classifiers import KerasClassifier from art.utils import load_dataset

Configure a logger to capture ART outputs; these are printed in console and the level of detail is set to INFO

logger = logging.getLogger() logger.setLevel(logging.INFO) handler = logging.StreamHandler() formatter = logging.Formatter("[%(levelname)s] %(message)s") handler.setFormatter(formatter) logger.addHandler(handler)

Read CIFAR10 dataset

(x_train, y_train), (x_test, ytest), min, max_ = load_dataset(str("cifar10")) im_shape = x_train[0].shape

Create Keras convolutional neural network - basic architecture from Keras examples

Source here: https://github.com/keras-team/keras/blob/master/examples/cifar10_cnn.py

model = Sequential() model.add(Conv2D(32, (3, 3), padding="same", input_shape=x_train.shape[1:])) model.add(Activation("relu")) model.add(Conv2D(32, (3, 3))) model.add(Activation("relu")) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Dropout(0.25))

model.add(Conv2D(64, (3, 3), padding="same")) model.add(Activation("relu")) model.add(Conv2D(64, (3, 3))) model.add(Activation("relu")) model.add(MaxPooling2D(pool_size=(2, 2))) model.add(Dropout(0.25))

model.add(Flatten()) model.add(Dense(512)) model.add(Activation("relu")) model.add(Dropout(0.5)) model.add(Dense(10)) model.add(Activation("softmax"))

model.compile(loss="categorical_crossentropy", optimizer="adam", metrics=["accuracy"])

Create classifier wrapper

classifier = KerasClassifier(model=model, clipvalues=(min, max_)) classifier.fit(x_train, y_train, nb_epochs=20, batch_size=128)

Craft adversarial samples

logger.info("Create attack") adv_crafter = PixelAttack(classifier, th=None, es=0, targeted=False, verbose=False)

logger.info("Craft attack on training examples") x_train_adv = adv_crafter.generate(x_train) logger.info("Craft attack test examples") x_test_adv = adv_crafter.generate(x_test)

Evaluate the classifier on the adversarial samples

preds = np.argmax(classifier.predict(x_test_adv), axis=1) acc = np.sum(preds == np.argmax(y_test, axis=1)) / y_test.shape[0] logger.info("Classifier before adversarial training") logger.info("Accuracy on adversarial samples: %.2f%%", (acc * 100))

[akshayag]

Same code is for ThresholdAttack

[beat-buesser]

@akshayag Thank you very much, this script is very helpful!

@shashankkotyan It looks like that the implementations of PixelAttack and ThresholdAttack create an error as described above if these attacks cannot find an adversarial example. I have created a possible solution in pull request #384 . Could you please take a look if this solution is correct?

[shashankkotyan]

@beat-buesser I will take a look at the pull request.