Trusted-AI / adversarial-robustness-toolbox

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
https://adversarial-robustness-toolbox.readthedocs.io/en/latest/
MIT License
4.88k stars 1.17k forks source link

Incorrect Dimension Order for Image Poisoning Perturbations #2045

Closed f4str closed 1 year ago

f4str commented 1 year ago

Describe the bug For all functions in adversarial-robustness-toolbox/blob/main/art/attacks/poisoning/perturbations/image_perturbations.py, the height and width dimensions are swapped. For Numpy, TensorFlow, and PyTorch the dimension order is height x width. However, for Pillow images, the order is width x height.

This causes a bug when using the insert_image function in ART as this will cause the height and width to be swapped. Since examples have only used square images, this bug was never found before.

To Reproduce When attempting to insert an image trigger for non-square images using the insert_image function, the height and width become flipped. Here is a simple example that can be run from the console.

>>> import numpy as np
>>> from art.attacks.poisoning.perturbations import insert_image
>>> x1 = np.random.rand(100, 50, 3)
>>> x2 = insert_image(x, backdoor_path='./utils/data/backdoors/alert.png', size=(8, 8), mode='RGB')
>>> x1.shape
(100, 50, 3)
>>> x2.shape
(50, 100, 3)

Additionally, here is further proof that Pillow uses a different dimension order.

>>> import numpy as np
>>> from PIL import Image
>>> x = Image.new("RGB", (100, 50))
>>> x.size
(100, 50)
>>> np.asarray(x).shape
(50, 100, 3)

Expected behavior In adversarial-robustness-toolbox/blob/main/art/attacks/poisoning/perturbations/image_perturbations.py, all instances of height and width should be swapped to the correct numpy order.

Screenshots N/A

System information (please complete the following information):

beat-buesser commented 1 year ago

Hi @f4str Thank you for catching and fixing this bug!