Implement Robust DPatch Attack

twosixlabs / armory-library

Python library for Adversarial ML Evaluation

https://twosixlabs.github.io/armory-library/

MIT License

9 stars 3 forks source link

Implement Robust DPatch Attack #149

Open deprit opened 7 months ago

deprit commented 7 months ago

Armory Library must support custom evasion attacks designed by the user; supporting user-developed attacks requires the steps below.

Design and implement a custom attack API that
- Optimizes an attack over a training split and, possibly, a validation set;
- Exposes the optimization loop to PyTorch Lightning;
- Logs optimization metrics and attack artifiacts to MLFlow.
Demonstrate a custom attack that
- Implements Robust DPatch for object detection (ART implementation);
- Optimizes the Robust DPatch attack on the VisDrone dataset using a pre-trained Hugging Face object detector.

treubig26 commented 5 months ago

This has been partially addressed by PR https://github.com/twosixlabs/armory-library/pull/155, however more work remains to generalize and integrate the attack optimization as part of the armory-library API.

Specifically, a new attack optimization engine should be created that will handle instantiation of the lightning module (with more hooks/parameterization) and the trainer.

The following comments from the PR should also be addressed in a follow-on PR: