Examples for Object Detection

[wuhanstudio]

Is your feature request related to a problem? Please describe. Hi, I notice that Robust DPatch is available in ART which is quite interesting, but the object detector is not implemented.

https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/art/attacks/evasion/dpatch_robust.py

https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/d351b3d33c266b436abbf13b9279f266a3dd3062/art/estimators/object_detection/object_detector.py#L39

I'm wondering will there be any examples of DPatch attacks against object detector.

Describe the solution you'd like

Perhaps a similar notebook such as the adversarial patch against a classifier.

https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/notebooks/adversarial_patch/attack_adversarial_patch.ipynb

[beat-buesser]

Hi @wuhanstudio Thank you very much for using ART!

This module adversarial-robustness-toolbox/art/estimators/object_detection/object_detector.py only contains the abstract base class for object detectors. Two model-specific object detection estimators are currently available in art.estimators.object_detection: PyTorchFasterRCNN and TensorFlowFasterRCNN.

We have currently a get-started example script for RobustDPatch and PyTorchFasterRCNN in https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/examples/get_started_fasterrcnn.py

[wuhanstudio]

Thanks for your help. I'll check this out and hope there will be YOLO support in the future.

[beat-buesser]

@wuhanstudio I agree, YOLO support would be great and it shouldn't be too difficult to implement. Let me know if you are interested to get start with an estimator for YOLO, we would be happy to help and provide support.

[wuhanstudio]

I have implemented several physical patch attacks against YOLO (Keras Model).

Perhaps I can add a file such as tensorflow_yolo.py after finishing my experiments soon.

[wuhanstudio]

[wuhanstudio]

I notice that ART imports models either from the official pytorch torchvision.models.detection.fasterrcnn_resnet50_fpn or the official tensorflow models.

Unfortunately, YOLO is officially supported by none the two:

• pytorch (https://pytorch.org/vision/stable/models.html)
• tensorflow (https://github.com/tensorflow/models/tree/master/research/object_detection/models)