Adversarial-Attacks-PyTorch

Torchattacks is a PyTorch library that provides adversarial attacks to generate adversarial examples.

It contains PyTorch-like interface and functions that make it easier for PyTorch users to implement adversarial attacks.

import torchattacks
atk = torchattacks.PGD(model, eps=8/255, alpha=2/255, steps=4)
# If inputs were normalized, then
# atk.set_normalization_used(mean=[...], std=[...])
adv_images = atk(images, labels)

Additional Recommended Packages.

• MAIR: Adversarial Trainining Framework, NeurIPS'23 Main Track.
• RobustBench: Adversarially Trained Models & Benchmarks, NeurIPS'21 Datasets and Benchmarks Track.

Citation. If you use this package, please cite the following BibTex (GoogleScholar):

@article{kim2020torchattacks,
title={Torchattacks: A pytorch repository for adversarial attacks},
author={Kim, Hoki},
journal={arXiv preprint arXiv:2010.01950},
year={2020}
}

:hammer: Requirements and Installation

Requirements

• PyTorch version >=1.4.0
• Python version >=3.6

Installation

#  pip
pip install torchattacks

#  source
pip install git+https://github.com/Harry24k/adversarial-attacks-pytorch.git

#  git clone
git clone https://github.com/Harry24k/adversarial-attacks-pytorch.git
cd adversarial-attacks-pytorch/
pip install -e .

:rocket: Getting Started

Precautions

• All models should return ONLY ONE vector of (N, C) where C = number of classes. Considering most models in torchvision.models return one vector of (N,C), where N is the number of inputs and C is thenumber of classes, torchattacks also only supports limited forms of output. Please check the shape of the model’s output carefully.
• The domain of inputs should be in the range of [0, 1]. Since the clipping operation is always applied after the perturbation, the original inputs should have the range of [0, 1], which is the general settings in the vision domain.
• torch.backends.cudnn.deterministic = True to get same adversarial examples with fixed random seed. Some operations are non-deterministic with float tensors on GPU [discuss]. If you want to get same results with same inputs, please run torch.backends.cudnn.deterministic = True[ref].

Demos

• Targeted mode

  • Random target label

    # random labels as target labels.
    atk.set_mode_targeted_random()

  • Least likely label

    # labels with the k-th smallest probability as target labels.
    atk.set_mode_targeted_least_likely(kth_min)

  • By custom function

    # labels obtained by mapping function as target labels.
    # shift all class loops one to the right, 1=>2, 2=>3, .., 9=>0
    atk.set_mode_targeted_by_function(target_map_function=lambda images, labels:(labels+1)%10)

  • By label

    atk.set_mode_targeted_by_label(quiet=True)
    # shift all class loops one to the right, 1=>2, 2=>3, .., 9=>0
    target_labels = (labels + 1) % 10
    adv_images = atk(images, target_labels)

  • Return to default

    atk.set_mode_default()

• Save adversarial images

  # Save
  atk.save(data_loader, save_path="./data.pt", verbose=True)
  
  # Load
  adv_loader = atk.load(load_path="./data.pt")

• Training/Eval during attack

  # For RNN-based models, we cannot calculate gradients with eval mode.
  # Thus, it should be changed to the training mode during the attack.
  atk.set_model_training_mode(model_training=False, batchnorm_training=False, dropout_training=False)

• Make a set of attacks

  • Strong attacks

    atk1 = torchattacks.FGSM(model, eps=8/255)
    atk2 = torchattacks.PGD(model, eps=8/255, alpha=2/255, iters=40, random_start=True)
    atk = torchattacks.MultiAttack([atk1, atk2])

  • Binary search for CW

    atk1 = torchattacks.CW(model, c=0.1, steps=1000, lr=0.01)
    atk2 = torchattacks.CW(model, c=1, steps=1000, lr=0.01)
    atk = torchattacks.MultiAttack([atk1, atk2])

  • Random restarts

    atk1 = torchattacks.PGD(model, eps=8/255, alpha=2/255, iters=40, random_start=True)
    atk2 = torchattacks.PGD(model, eps=8/255, alpha=2/255, iters=40, random_start=True)
    atk = torchattacks.MultiAttack([atk1, atk2])

:page_with_curl: Supported Attacks

The distance measure in parentheses.

:bar_chart: Performance Comparison

As for the comparison packages, currently updated and the most cited methods were selected:

• Foolbox: 611 citations and last update 2023.10.
• ART: 467 citations and last update 2023.10.

Robust accuracy against each attack and elapsed time on the first 50 images of CIFAR10. For L2 attacks, the average L2 distances between adversarial images and the original images are recorded. All experiments were done on GeForce RTX 2080. For the latest version, please refer to here (code, nbviewer).

Note that Foolbox returns accuracy and adversarial images simultaneously, thus the actual* time for generating adversarial images might be shorter than the records.

^†Considering that the binary search algorithm for const c can be time-consuming, torchattacks supports MutliAttack for grid searching c.

Additionally, I also recommend to use a recently proposed package, Rai-toolbox.

The rai-toolbox takes a unique approach to gradient-based perturbations: they are implemented in terms of parameter-transforming optimizers and perturbation models. This enables users to implement diverse algorithms (like universal perturbations and concept probing with sparse gradients) using the same paradigm as a standard PGD attack.