search

locuslab / convex_adversarial

A method for training neural networks that are provably robust to adversarial attacks.
MIT License
377 stars 84 forks source link

it is possible to change conv2d's padding to zero? #15

Open cxmscb opened 5 years ago

cxmscb commented 5 years ago

After changing the padding of conv2d to zero, error occurred below：

File "fashion_mnist.py", line 59, in args.alpha_grad, args.scatter_grad, l1_proj=args.l1_proj) File "/home/songcb/convex_adversarial-master/examples/trainer.py", line 22, in train_robust scatter_grad=scatter_grad) File "/home/songcb/.local/lib/python3.6/site-packages/convex_adversarial-0.2-py3.6.egg/convex_adversarial/dual.py", line 182, in robust_loss File "/home/songcb/.local/lib/python3.6/site-packages/convex_adversarial-0.2-py3.6.egg/convex_adversarial/dual.py", line 56, in init File "/home/songcb/.local/lib/python3.6/site-packages/convex_adversarial-0.2-py3.6.egg/convex_adversarial/affine.py", line 46, in call File "/home/songcb/.local/lib/python3.6/site-packages/convex_adversarial-0.2-py3.6.egg/convex_adversarial/affine.py", line 66, in forward File "/home/songcb/.local/lib/python3.6/site-packages/torch/nn/functional.py", line 837, in linear output = input.matmul(weight.t()) File "/home/songcb/.local/lib/python3.6/site-packages/torch/autograd/variable.py", line 386, in matmul return torch.matmul(self, other) File "/home/songcb/.local/lib/python3.6/site-packages/torch/functional.py", line 173, in matmul return torch.mm(tensor1, tensor2) RuntimeError: size mismatch at /pytorch/torch/lib/THC/generic/THCTensorMathBlas.cu:243

riceric22 commented 5 years ago

Hey @cxmscb,

Yes, it should be possible, but there is likely a size mismatch bug due to the transpose operator needing an output padding, e.g. the same issue that comes up here: https://github.com/pytorch/pytorch/issues/5057. I suspect this can be fixed by passing output_padding=1 to the transpose convolution, can you share what your stride length and input sizes are?

cxmscb commented 5 years ago

Thanks ! The stride of conv is 2, and the input size is 28x28.

riceric22 commented 5 years ago

The quick and dirty way to fix this would be to add output_padding=1 to the transpose convolution call. So if you're using v0.2, on line 119 of convex_adversarial/affine.py (which it appears to be the case in your trace), change 

        out = conv_transpose2d(x.contiguous(),  self.l.weight, 
                                 stride=self.l.stride,
                                 padding=self.l.padding)

to 

        out = conv_transpose2d(x.contiguous(),  self.l.weight, 
                                 stride=self.l.stride,
                                 padding=self.l.padding, 
                                 output_padding=1)

Alternatively, if you're using the master branch, then the same line (and same fix) can be found on line 143 of convex_adversarial/dual_layers.py.

In the ideal case, we should calculate when the output padding is necessary given input size, stride length, and padding, but how to do this in general is not quite clear to me yet, so I will leave this issue open.

cxmscb commented 5 years ago

Thanks!!!