csdongxian / skip-connections-matter

Codes for ICLR 2020 paper "Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets"
MIT License
70 stars 9 forks source link
iclr skip-connections

Skip Connections Matter

This repository contains the code for Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets (ICLR 2020 Spotlight).

News

Method

We propose the Skip Gradient Method (SGM) to generate adversarial examples using gradients more from the skip connections rather than the residual modules. In particular, SGM utilizes a decay factor (gamma) to reduce gradients from the residual modules,

Requisite

This code is implemented in PyTorch, and we have tested the code under the following environment settings:

Run the code

  1. Download the dataset from Google Drive or Baidu Drive (pw:55rk), and extract images to the path ./SubImageNet224/
  2. Generate adversarial examples and save them into path ./adv_images/. For ResNet-152 as the source model,
    python attack_sgm.py --gamma 0.2 --output_dir adv_images --arch densenet201 --batch-size 40

    For DenseNet-201 as the source model,

    python attack_sgm.py --gamma 0.5 --output_dir adv_images --arch resnet152 --batch-size 40
  3. Evaluate the transerability of generated adversarial examples in ./adv_images/. For VGG19 with batch norm as the target model
    python evaluate.py --input_dir adv_images --arch vgg19_bn

Results

We run this code, and the attack success (1 - acc) against VGG19 is close to the repored in our paper:

Source \ Method PGD MI SGM
ResNet-152 45.80% 66.70% 81.04%
DenseNet-201 57.82% 75.38% 82.58%

Implementation

For easier reproduction, we provide more detailed information here.

register backward hook for SGM

In fact, we manipulate gradients flowing through ReLU in utils_sgm, since there is no ReLU in skip-connections:

Pretrained models

All pretrained models in our paper can be found online:

Citing this work

@inproceedings{wu2020skip,
    title={Skip connections matter: On the transferability of adversarial examples generated with resnets},
    author={Wu, Dongxian and Wang, Yisen and Xia, Shu-Tao and Bailey, James and Ma, Xingjun},
    booktitle={ICLR},
    year={2020}
}

Reference

[1] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In CVPR, 2018.

[2] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, Patrick McDaniel. Ensemble Adversarial Training: Attacks and Defenses. In ICLR, 2018.