Trusted-AI / adversarial-robustness-toolbox

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
https://adversarial-robustness-toolbox.readthedocs.io/en/latest/
MIT License
4.82k stars 1.16k forks source link

EnsembleClassifier not receiving `raw` parameter from attack #214

Closed dapello closed 4 years ago

dapello commented 4 years ago

Describe the bug EnsembleClassifier expects a parameter raw for predict, loss_gradient, and class_gradient, which is not passed by ProjectedGradientDescent or HopSkipJump generate functions.

To Reproduce Steps to reproduce the behavior:

import torch
from torchvision import models

from art.classifiers import PyTorchClassifier, EnsembleClassifier
from art.attacks import ProjectedGradientDescent, HopSkipJump

# load and preprocess imagenet images
images = load_preprocess_images(...)

resnet18 = models.resnet18(pretrained=True)
alexnet = models.alexnet(pretrained=True)

wrapped_models = []
for model in [resnet18, alexnet]:
    loss = torch.nn.CrossEntropyLoss()
    optimizer = torch.optim.SGD(model.parameters(), .1, momentum=0.9, weight_decay=1e-4)
    wrapped_model = PyTorchClassifier(model=model, loss=loss, optimizer=optimizer, input_shape=(3,224,224), nb_classes=1000)
    wrapped_models.append(wrapped_model)

ensemble = EnsembleClassifier(wrapped_models)
attack = ProjectedGradientDescent(ensemble)
adv_images = attack.generate(x = images)

"""
yields
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-117-766010f0b586> in <module>
      1 attack = ProjectedGradientDescent(ensemble, eps=10000)
----> 2 adv_image = attack.generate(x = images)

~/anaconda3/envs/brainscore/lib/python3.6/site-packages/art/attacks/projected_gradient_descent.py in generate(self, x, y, **kwargs)
    108 
    109             # Use model predictions as correct outputs
--> 110             targets = get_labels_np_array(self.classifier.predict(x, batch_size=self.batch_size))
    111         else:
    112             targets = y

~/anaconda3/envs/brainscore/lib/python3.6/site-packages/art/classifiers/ensemble.py in predict(self, x, batch_size, **kwargs)
    114             raw = kwargs['raw']
    115         else:
--> 116             raise ValueError('Missing argument `raw`.')
    117 
    118         preds = np.array([self._classifier_weights[i] * self._classifiers[i].predict(x)

ValueError: Missing argument `raw`.

"""

Expected behavior To generate adversarial images based on the ensemble of classifiers.

System information (please complete the following information):

beat-buesser commented 4 years ago

Hi @dapello Thank you very much for using ART and raising this issue! We will take a look at it as soon as possible.

beat-buesser commented 4 years ago

I think we need to add a default value of False for the keyword argument raw in predict, loss_gradient, and class_gradient in ensemble classifier instead of raising ValueError.

dapello commented 4 years ago

Looking at the source, that seems like a good fix -- happy to PR if you like.

beat-buesser commented 4 years ago

That would be great! I'll assign this issue to you. Please start from branch dev_1.1.0 and let me know if you have any questions.

dapello commented 4 years ago

This is working perfectly now, thanks for taking my PR!