About BN in Advprop

[lucasliunju]

I would like to ask whether PGD attacker (calculate gradients) will update the parameters (mean, var) of aux_bn in Resnet. Thank you.

[tingxueronghua]

Sorry about forgetting to reply. No, in this implementation, PGD attacker won't update the parameters of aux_bn during attacking. In fact, this is a problem about which mode of batch normalization is better to use during attacking. As for our experience on ImageNet, eval mode is better, which freezes the parameters (mean, var) of batch normalization. Similar results are reported on Cifar by Tianyu et al. in "BAG OF TRICKS FOR ADVERSARIAL TRAINING".