Model outputs significantly different depending on batch size and contents

[carlini]

Thank you very much for releasing the model and associated code along with your paper. I'm very grateful that you've put the effort into making it as easy as possible to get everything up and running, and I sincerely hope others involved in the contest follow your lead.

I'm taking an initial first pass at looking at everything, and am getting somewhat confusing results. First, it looks like the model is giving very different results at small batch sizes:

from models.small_cnn import SmallCNN
import numpy as np
import torch
from models.wideresnet import WideResNet

from torch.autograd import Variable
import torch.optim as optim
import torch.nn as nn

device = torch.device("cuda")
model = WideResNet().to(device)
model.load_state_dict(torch.load('./checkpoints/model_cifar_wrn.pt'))

X_data = np.load("data_attack/cifar10_X.npy")
Y_data = np.load("data_attack/cifar10_Y.npy")
X_data = np.transpose(X_data, (0, 3, 1, 2))

for bs in [1, 2, 4, 5, 10, 50, 100]:
    predictions = []
    for i in range(0,100,bs):
        logits = model(torch.from_numpy(np.array(X_data[i:i+bs], dtype=np.float32)).to(device)).cpu().detach().numpy()
        predictions.extend(np.argmax(logits,axis=1))
    print("mean accuracy with batch size %d: %f"%(bs,np.mean(predictions == Y_data[:100])))

will output

mean accuracy with batch size 1: 0.140000
mean accuracy with batch size 2: 0.640000
mean accuracy with batch size 4: 0.760000
mean accuracy with batch size 5: 0.790000
mean accuracy with batch size 10: 0.840000
mean accuracy with batch size 50: 0.840000
mean accuracy with batch size 100: 0.830000

It looks like there is also some dependence on the data of the batch to classify each input. I have some batch of 10000 examples I want to process as a [100, 100, 3, 32, 32] matrix, and if I process them in row-major order I get a different accuracy than column-major. I suspect this might have the same underlying cause, so I'll give details for that later if necessary.

As you might imagine, this makes it difficult to evaluate the defense: evaluating the network with a batch of [99 clean examples] + [1 adversarial example] gives a different result than [50 clean examples] + [50 adversarial examples].

Is this intended, am I doing something wrong, or something else?

[hongyanz]

Dear Nicholas,

Thanks so much for trying our defense method.

We tested your code just now and found that once 'model.eval()' is added in the code, the issue will be fixed. This is simply because WideResNet has the batch-normalization operator, and model.eval() will help to fix the batch normalization during the test time. Please let us know if you have further question. Thanks again.

Best, Hongyang

[carlini]

Ah, well this is where my lack of PyTorch knowledge really shows. Thanks.