BackdoorBox: An Open-sourced Python Toolbox for Backdoor Attacks and Defenses

Backdoor attacks are emerging yet critical threats in the training process of deep neural networks (DNNs), where the adversary intends to embed specific hidden backdoor into the models. The attacked DNNs will behave normally in predicting benign samples, whereas the predictions will be maliciously changed whenever the adversary-specified trigger patterns appear. Currently, there were many existing backdoor attacks and defenses. Although most of them were open-sourced, there is still no toolbox that can easily and flexibly implement and compare them simultaneously.

BackdoorBox is an open-sourced Python toolbox, aiming to implement representative and advanced backdoor attacks and defenses under a unified framework that can be used in a flexible manner. We will keep updating this toolbox to track the latest backdoor attacks and defenses.

Currently, this toolbox is still under development (but the attack parts are almost done) and there is no user manual yet. However, users can easily implement our provided methods by referring to the tests sub-folder to see the example codes of each implemented method. Please refer to our paper for more details! In particular, you are always welcome to contribute your backdoor attacks or defenses by pull requests!

Toolbox Characteristics

• Consistency: Instead of directly collecting and combining the original codes from each method, we re-implement all methods in a unified manner. Specifically, variables having the same function have a consistent name. Similar methods inherit the same base class for further development, have a unified workflow, and have the same core sub-functions (e.g., get_model()).
• Simplicity: We provide code examples for each implemented backdoor attack and defense to explain how to use them, the definitions and default settings of all required attributes, and the necessary code comments. Users can easily implement and develop our toolbox.
• Flexibility: We allow users to easily obtain important intermediate outputs and components of each method (e.g., poisoned dataset and attacked/repaired model), use their local samples and model structure for attacks and defenses, and interact with their local codes. The attack and defense modules can be used jointly or separately. You can also use your local dataset via torchvision.datasets.DatasetFolder. (See examples of using the GTSRB dataset)
• Co-development: All codes and developments are hosted on Github to facilitate collaboration. Currently, there are more than seven contributors have helped develop the code base and others have contributed to the code test. This developing paradigm facilitates rapid and comprehensive development and bug finding.

Backdoor Attacks

Note: For the convenience of users, all our implemented attacks support obtaining poisoned dataset (via .get_poisoned_dataset()), obtaining infected model (via .get_model()), and training with your own local samples (loaded via torchvision.datasets.DatasetFolder). Please refer to base.py and the attack's codes for more details.

Backdoor Defenses

Methods Under Development

• DBD
• SS
• Neural Cleanse
• DP
• CutMix
• AEVA
• STRIP

Attack & Defense Benchmark

The benchmark is coming soon.

Contributors

Citation

If our toolbox is useful for your research, please cite our paper(s) as follows:

@inproceedings{li2023backdoorbox,
  title={{BackdoorBox}: A Python Toolbox for Backdoor Learning},
  author={Li, Yiming and Ya, Mengxi and Bai, Yang and Jiang, Yong and Xia, Shu-Tao},
  booktitle={ICLR Workshop},
  year={2023}
}

@article{li2022backdoor,
  title={Backdoor learning: A survey},
  author={Li, Yiming and Jiang, Yong and Li, Zhifeng and Xia, Shu-Tao},
  journal={IEEE Transactions on Neural Networks and Learning Systems},
  year={2022}
}