search

cure-lab / MMA-Diffusion

[CVPR2024] MMA-Diffusion: MultiModal Attack on Diffusion Models
Other
85 stars 5 forks source link

Question about image attack #6

Open jiajiaxiaoskx opened 1 week ago

jiajiaxiaoskx commented 1 week ago

Your work is particularly amazing! I have two questions I’d like to ask.

  1. In your image attack code, you commented out the mask section, causing some variables to be referenced without being defined. How should I modify the code to run it properly?

    image

  2. What is the purpose of the safetychecker.pt file in the image_space_attack folder in the GitHub repository, and how does it differ from the safety_checker.pt from Hugging Face?

Thank you!

yangyijune commented 1 week ago

Hi! For the first question, you can directly download the mask and image .png files from https://huggingface.co/datasets/YijunYang280/MMA_Diffusion_adv_images_benchmark

Regarding the .pt files, both are identical. We provided them to make it easier for users to employ our attack methods. You can also find the same safety checker in the SDv1.5 checkpoint by yourself.

jiajiaxiaoskx commented 5 days ago

thanks! When I was running image attack code, another error came. attack successful top1 rate: 0.75 Traceback (most recent call last): File "/data4/skx/MMA-Diffusion/src/image_space_attack/image_editing_attack.py", line 528, in main(args) File "/data4/skx/MMA-Diffusion/src/image_space_attack/image_editing_attack.py", line 476, in main images_test = processor(img)[None] File "/data3/skx/miniconda3/envs/mma/lib/python3.10/site-packages/torchvision/transforms/transforms.py", line 95, in call img = t(img) File "/data3/skx/miniconda3/envs/mma/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/data3/skx/miniconda3/envs/mma/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, **kwargs) File "/data3/skx/miniconda3/envs/mma/lib/python3.10/site-packages/torchvision/transforms/transforms.py", line 980, in forward return F.resized_crop(img, i, j, h, w, self.size, self.interpolation, antialias=self.antialias) File "/data3/skx/miniconda3/envs/mma/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1695, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'") AttributeError: 'RandomResizedCrop' object has no attribute 'antialias'

I do not if the error is related to the "open_clip_preprocess.pt" or the version of torchvision. Thanks!

yangyijune commented 2 days ago

It seems there is a package conflict issue. The file open_clip_preprocess.pt is quite simple, containing only normalization parameters (i.e., mean and variance). If changing the version does not resolve the issue, you can rewrite this part by following OpenClip's preprocessing code.