JonasGeiping / invertinggradients

Algorithms to recover input data from their gradient signal through a neural network
MIT License
275 stars 71 forks source link

Is this method suitable for attacking the vit model? #10

Closed Amazingren closed 2 years ago

Amazingren commented 2 years ago

Hey @JonasGeiping,

Hoping everything goes well. And thanks for your impressive work. I wonder if this model is suitable for attacking the vit model? I would appreciate it if you can shed some light on this.

Many thanks,

JonasGeiping commented 2 years ago

Hi, this attack is generic and can be tried against any model architecture.

However, recent attacks that specialize on attacks against vision transformers such as APRIL https://arxiv.org/abs/2112.14087 or gradVIT https://arxiv.org/abs/2203.11894 are likely to be even more effective.

P.S: You can find our open-source implementation of the first attack here: https://github.com/JonasGeiping/breaching/blob/main/examples/APRIL%20%20-%20Analytic%20Attack%20-%20Vision%20Transformer%20on%20ImageNet.ipynb

Amazingren commented 2 years ago

Hi, this attack is generic and can be tried against any model architecture.

However, recent attacks that specialize on attacks against vision transformers such as APRIL https://arxiv.org/abs/2112.14087 or gradVIT https://arxiv.org/abs/2203.11894 are likely to be even more effective.

P.S: You can find our open-source implementation of the first attack here: https://github.com/JonasGeiping/breaching/blob/main/examples/APRIL%20%20-%20Analytic%20Attack%20-%20Vision%20Transformer%20on%20ImageNet.ipynb

Thanks for your quick and patient reply, your suggestion helps a lot! I will try the APRIL and let you know if I have any further questions~