Trusted-AI / adversarial-robustness-toolbox

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
https://adversarial-robustness-toolbox.readthedocs.io/en/latest/
MIT License
4.78k stars 1.16k forks source link

AssertionError: assert grds.shape == (x_.shape[0], 1) + self.input_shape #21

Closed JindongGu closed 5 years ago

JindongGu commented 5 years ago

Hi all, I apply CarliniL2Method(classifier...).generate(img_variable.data.numpy()) to generate asversary samples. It shows that Traceback (most recent call last): File "adv_tool.py", line 127, in adv_gen() File "adv_tool.py", line 113, in adv_gen img_adv = adv.generate(img_variable.data.numpy()) File "/Users/ostdong/anaconda2/lib/python2.7/site-packages/art/attacks/carlini.py", line 184, in generate grad_l2p = self.classifier.class_gradient(np.array([adv_image]), label=i_add, logits=True)[0] File "/Users/ostdong/anaconda2/lib/python2.7/site-packages/art/classifiers/pytorch.py", line 216, in classgradient assert grds.shape == (x.shape[0], 1) + self.input_shape AssertionError

The classifier I used is Pytorchclassifier. It works for other methods like FastGradientMethod. img_variable.data.numpy() has shape (1, 3, 224, 224), as stated in the classifier arguments. The same error with DeepFool method (both targeted and untargeted). anybody knows why? Thanks!

ririnicolae commented 5 years ago

Hi, @Jindong-Explainable-AI! The fact that you get the error for C&W and DeepFool, but not FGSM confirms that the problem comes from the implementation of class gradients for the PyTorch wrapper. A couple of questions:

JindongGu commented 5 years ago

Hi, @Irina Nicolae Thanks for the response! I reinstalled the package again with pip command and also manually. And run the code, get the same error! I also tried with different (input_shape) e.g. (224, 224), (1, 224, 224), (1, 3, 224, 224), (3, 224, 224), (224, 224, 3) and so on, still the same error.

JindongGu commented 5 years ago

model = models.vgg16(pretrained=True)

classifier = PyTorchClassifier(clip_values=(-2.11790394, 2.2489084), model=model, loss= nn.CrossEntropyLoss(), optimizer= torch.optim.SGD(model.parameters(), lr = 0.01), input_shape=(3, 244, 244), nb_classes = 1000, defences = None, preprocessing=(0, 1))

adv = CarliniL2Method(classifier, targeted=False, max_iter=10, binary_search_steps=10, learning_rate=0.01)

adv.generate(img_variable.data.numpy())

the shape of img_variable.data.numpy() is (1, 3, 224, 224)

FYI to reproduce the error!

ririnicolae commented 5 years ago

Hi, @Jindong-Explainable-AI! I managed to reproduce your error. In the code you posted, there is an error in input_shape when you create the PyTorchClassifier object: you use (3, 244, 244) instead of (3, 224, 224). :smile: Correcting these values will allow you to use the attacks.

JindongGu commented 5 years ago

Thanks for this! Sorry for such an embarrassing error. FastGradientMethod, BIM use loss_gradient() where assert grds.shape == x.shape In CarliniL2Method, it uses classgradient() where assert grds.shape == (x.shape[0], self.nb_classes) + self.input_shape The first one has nothing to do with the specified input shape. That is why one works, it does not work for the other! Okay, Thanks a lot!