pytorch_ares

This repository contains the code for ARES (Adversarial Robustness Evaluation for Safety), a Python library for adversarial machine learning research focusing on benchmarking adversarial robustness on image classification correctly and comprehensively.

Installation

• Clone this repo

  git clone https://github.com/haichen-ber/pytorch_ares.git

• Install the experimental environment

  pip install -r requirements.txt

  The requirements.txt includes its dependencies.

  Files in the folder

• pytorch_ares/
  • data/: The code supports cifar10 and imagenet datasets.
  • test/: Some toyexamples for testing adversarial attack methods and adversarial defense methods.
  • pytorch_ares/
  • dataset_torch/: Data processing for cifar10 and imagenet datasets.
  • attack_torch/: PyTorch implementation of some adversarial attack methods.
  • cifar10_model/: PyTorch implementation of some adversarial defense models on the cifar10 dataset.
  • defense_torch/: PyTorch implementation of some defense methods.
  • third_party/: Other open source repositories.
  • attack_benchmark/: Adversarial robustness benchmarks for image classification.

    Supported Methods

Adversarial attack

• FGSM: Explaining and harnessing adversarial examples
• BIM: Adversarial examples in the physical world
• PGD: Towards Deep Learning Models Resistant to Adversarial Attacks
• CW: Towards Evaluating the Robustness of Neural Networks
• DeepFool: DeepFool: a simple and accurate method to fool deep neural networks
• MIM: Boosting Adversarial Attacks with Momentum
• DIM: Improving Transferability of Adversarial Examples with Input Diversity
• TIM: Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks
• SI-NI-FGSM: Nesterov accelerated gradient and scale invariance for adversarial attacks
• VIM: Enhancing the Transferability of Adversarial Attacks through Variance Tuning
• SGM: Skip connections matter: On the transferability of adversarial examples generated with resnets
• CDA: Cross-Domain Transferability of Adversarial Perturbations
• AutoAttack: Reliable Evaluation of Adversarial Robustness with an Ensemble of Diverse Parameter-free Attacks
• Boundary: Decision-based adversarial attacks: Reliable attacks against black-box machine learning models
• SPSA: Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• Evolutionary: Efficient Decision-based Black-box Adversarial Attacks on Face Recognition
• NES: Black-box Adversarial Attacks with Limited Queries and Information
• Nattack: NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
• TTA: On Success and Simplicity: A Second Look at Transferable Targeted Attacks

Adversarial defense

• RST: Unlabeled Data Improves Adversarial Robustness

• TRADES: Theoretically Principled Trade-off between Robustness and Accuracy

• FS-AT: Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training

• Pre-Training: Using Pre-Training Can Improve Model Robustness and Uncertainty

• AT-HE: Boosting Adversarial Training with Hypersphere Embedding

• Robust Overfitting: Overfitting in adversarially robust deep learning

• FastAT: Fast is better than free: Revisiting adversarial training

• AWP: Adversarial Weight Perturbation Helps Robust Generalization

• HYDRA: HYDRA: Pruning Adversarially Robust Neural Networks

• Label Smoothing: Bag of Tricks for Adversarial Training

Example to run the codes

ARES provides command line interface to run benchmarks. For example, you can test the attack success rate of fgsm on resnet18 on the cifar10 dataset:

cd test/
python test_fgsm.py --dataset_name cifar10

There are 4 run_***.py files in the attack_benchmark folder that evaluate the adversarial robustness benchmarks on the cifar10 and imagenet datasets. For example, if you want to evaluate the robustness of the defense model on the cifar10 dataset, you can run the following command line:

cd attack_benchmark/ 
python run_cifar10_defense.py