cure-lab / MMA-Diffusion

[CVPR2024] MMA-Diffusion: MultiModal Attack on Diffusion Models
Other
85 stars 5 forks source link

Questions about the paper regarding text embedding adversary attack #2

Open LezJ opened 1 month ago

LezJ commented 1 month ago

Hi authors, thanks for your amazing work to bring the existing safety problem of T2I models. I assume that your text embedding based adversary attack requires to access the exact text encoder of the model right? But in real life settings, normally a user won't know the exact type of text encoder using (for example, if you are using Dalle3). I guess we cant apply the attack in such situations right?

Best

yangyijune commented 1 month ago

The proposed attacks are conducted on the open-source Stable Diffusion, then directly transfered to attack other type T2I models. We found the transfer attack success rate is decent. Though text encoders within different T2I models have distinct architecture, they may trained on same language materials, therefore capturing the similar underlying semantic relationships among words, resulting in the transferability.