search

ZiangYan / deepdefense.pytorch

Implementation of our NeurIPS 2018 paper: Deep Defense: Training DNNs with Improved Adversarial Robustness
39 stars 7 forks source link

Preprocessing #3

Closed kayween closed 3 years ago

kayween commented 5 years ago

Hi Ziangyan,

In the code you have used the same preprocessing as MatConvNet. Can you describe the preprocessing in detail? I assume that it is subtracting the mean, as written in the source code of MatConvNet. But it seems like it does not work.

ZiangYan commented 5 years ago

Hi kaiwenwu96,

I do not choose the default dataloader in PyTorch mainly because we need to make a fair comparison with the DeepFool paper (CVPR 2016).

I use the matconvnet mnist example data processing procedure: https://github.com/vlfeat/matconvnet/blob/6221423ac64067426bbaa2ffd31557523258d1fe/examples/mnist/cnn_mnist.m#L88-L138

Concretely, I just subtract the per-pixel mean, and no other pre-processing step is involved.

The per-pixel mean vector is also included in the mnist-data-0208ce21.mat file, please see the proof in the below matlab console:

mean

kayween commented 5 years ago

Hi Ziangyan,

Thanks for clarification! Can I also confirm that CIFAR10 use the same preprocessing method (subtracting the mean)? There is no mean vector in the .mat of CIFAR. And when I try to compute the mean on CIFAR, I found the range of the data seems weird. The maximum value and minimum value can exceed [-255, 255].

Selection_023

In addtion, I guess the reason why you subtracting mean is because the evaluation metric, perturbation ration, depends on the norm of data. It is not translation invariant. Thus you have to keep the range the same as deepfool.

ZiangYan commented 5 years ago

Hi,

MatConvNet adopts a different pre-processing approach in CIFAR-10, please see https://github.com/vlfeat/matconvnet/blob/master/examples/cifar/cnn_cifar.m#L93-L154 for more details.

opts.whitenData and opts.contrastNormalization are set to true by default, as in https://github.com/vlfeat/matconvnet/blob/master/examples/cifar/cnn_cifar.m#L18-L19

In fact, I first run DeepFool's official MATLAB implementation, and load their generated .mat data file into pytorch, since I want to make sure my pytorch implementation has an exact output with their code.