Closed mzweilin closed 2 years ago
@dxoigmn pointed out that the saturated colors in the example patch implies that the learning rate of the attack by default is too large.
After lowering the learning rate from 5
to 5/255
, the 10-iteration attack is more successful: mAP 44.67% -> 37.67%. We also share the results on 1000-iteration in the next comment.
$ jq ".attack.kwargs.learning_rate = 5/255 | .scenario.export_samples = 165" scenario_configs/carla_obj_det_dpatch_undefended.json | armory run --no-docker --use-gpu -
Evaluation: 100%|█| 165/165 [09:50<00:00, 3.58s/it]
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO carla_od_AP_per_class on benign test examples relative to ground truth labels: {1: 0.51000000000000001, 2: 0.40000000000000002, 3: 0.42999999999999999}
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO mean carla_od_AP_per_class on benign examples relative to ground truth labels 44.67%.
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO Average carla_od_disappearance_rate on benign test examples relative to ground truth labels: 43.72%
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO Average carla_od_hallucinations_per_image on benign test examples relative to ground truth labels: 0.81
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO Average carla_od_misclassification_rate on benign test examples relative to ground truth labels: 3.54%
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO Average carla_od_true_positive_rate on benign test examples relative to ground truth labels: 52.74%
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO carla_od_AP_per_class on adversarial test examples relative to ground truth labels: {1: 0.46000000000000002, 2: 0.38, 3: 0.28999999999999998}
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO mean carla_od_AP_per_class on adversarial examples relative to ground truth labels 37.67%.
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO Average carla_od_disappearance_rate on adversarial test examples relative to ground truth labels: 43.32%
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO Average carla_od_hallucinations_per_image on adversarial test examples relative to ground truth labels: 2.4
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO Average carla_od_misclassification_rate on adversarial test examples relative to ground truth labels: 3.67%
2021-11-22 09:30:30 spr-gpu04 armory.utils.metrics[94929] INFO Average carla_od_true_positive_rate on adversarial test examples relative to ground truth labels: 53.02%
2021-11-22 09:30:31 spr-gpu04 armory.scenarios.scenario[94929] INFO Saving evaluation results to path
/home/weilinxu/.armory/outputs/2021-11-22T172027.771428/CarlaObjectDetectionTask_1637602231.json
⭡
jq ".attack.kwargs.max_iter = 1000" scenario_configs/carla_obj_det_dpatch_undefended.json
⭡
jq ".attack.kwargs.learning_rate = 5/255" scenario_configs/carla_obj_det_dpatch_undefended.json
⭡
jq ".attack.kwargs.max_iter = 1000 | .attack.kwargs.learning_rate = 5/255" scenario_configs/carla_obj_det_dpatch_undefended.json
mAP 44.67% -> 15.67%
$ jq ".attack.kwargs.max_iter = 1000 | .attack.kwargs.learning_rate = 5/255 | .scenario.export_samples = 165" scenario_configs/carla_obj_det_dpatch_undefended.json
RobustDPatch iteration: 100%|█| 1000/1000 [05:34<00:00, 2.99it/s]
Evaluation: 100%|███████████| 165/165 [12:06:11<00:00, 264.07s/it]
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO carla_od_AP_per_class on benign test examples relative to ground truth labels: {1: 0.51000000000000001, 2: 0.40000000000000002, 3: 0.42999999999999999}
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO mean carla_od_AP_per_class on benign examples relative to ground truth labels 44.67%.
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO Average carla_od_disappearance_rate on benign test examples relative to ground truth labels: 43.72%
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO Average carla_od_hallucinations_per_image on benign test examples relative to ground truth labels: 0.81
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO Average carla_od_misclassification_rate on benign test examples relative to ground truth labels: 3.54%
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO Average carla_od_true_positive_rate on benign test examples relative to ground truth labels: 52.74%
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO carla_od_AP_per_class on adversarial test examples relative to ground truth labels: {1: 0.25, 2: 0.20000000000000001, 3: 0.02}
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO mean carla_od_AP_per_class on adversarial examples relative to ground truth labels 15.67%.
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO Average carla_od_disappearance_rate on adversarial test examples relative to ground truth labels: 44.98%
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO Average carla_od_hallucinations_per_image on adversarial test examples relative to ground truth labels: 1e+01
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO Average carla_od_misclassification_rate on adversarial test examples relative to ground truth labels: 3.36%
2021-11-22 21:26:06 spr-gpu04 armory.utils.metrics[94429] INFO Average carla_od_true_positive_rate on adversarial test examples relative to ground truth labels: 51.66%
2021-11-22 21:26:06 spr-gpu04 armory.scenarios.scenario[94429] INFO Saving evaluation results to path
/home/weilinxu/.armory/outputs/2021-11-22T171945.081514/CarlaObjectDetectionTask_1637645166.json
Based on the experimental results, I would suggest to add attack.kwargs.learning_rate = 5/255
in scenario_configs/carla_obj_det_dpatch_undefended.json
to better demonstrate attacks.
thanks for sharing these results @mzweilin! At the time of the most recent Armory release, we hadn't had time to properly tune attack parameters and there was the issue #1212 addresses. For visibility's sake I'll also tag @yusong-tan who has been tuning attacks. We will ensure that default configs are updated accordingly as part of the next Armory release preceding Eval 4.
I think the default learning_rate of 5 was an artifact of the original Dpatch implementation, which assumed input in the range of [0, 255]. Since Armory now normalizes all input to [0, 1], the learning rate will need to be normalized as well. I found that learning rate on the same order as 0.01 works well.
Closing due to #1230
Dear Armory Team,
I ran the scenario with 1000-iteration attacks, but the adversarial mAP is very close to the benign mAP (0.4367 over 0.4467). Shall we expect the adversarial mAP to be much lower than that, considering the target model is undefended?
BTW, the rectangular patch from https://github.com/twosixlabs/armory/pull/1212 is used.
0_benign.png![0_benign](https://user-images.githubusercontent.com/1704487/142903385-1443912a-17e5-4c9a-96b3-e01061f22c3e.png)
0_adversarial.png![0_adversarial](https://user-images.githubusercontent.com/1704487/142903408-a8437cfd-dc6e-49d4-a92d-19075ca091b9.png)