Verify specific features

[EmanueleLM]

Hi,

first of all, thank you for your work, I think it's amazing.

I've a question: let's suppose that I want to verify a model that has been trained on MNIST (let's suppose for simplicity that is one of the models you provide with the code), but I need just to verify robustness against some specific features (i.e. pixels): this means that an attacker is able to manipulate only them, while the others remain unchanged.

Is there any 'off-the-shelf' way to do that or, if not, how should I change the code?

Thank you, Emanuele

[AkhilanB]

Hi,

Thank you for your interest!

Unfortunately there is no off-the-shelf way to verify on only a specific set of features.

In the case of L__\infty perturbations, one way to modify the code may be to modify the eps argument of find_output_bounds(https://github.com/IBM/CNN-Cert/blob/master/cnn_bounds_full_core.py#L443, https://github.com/IBM/CNN-Cert/blob/master/cnn_bounds_full.py#L564) to be a tensor instead of a scalar with the allowable level of perturbation for each pixel (0 for unchanged pixels). This would require also modifying the conv_bound (https://github.com/IBM/CNN-Cert/blob/master/cnn_bounds_full_core.py#L199) and conv_bound_full (https://github.com/IBM/CNN-Cert/blob/master/cnn_bounds_full_core.py#L231) functions to use a tensor eps.

In the case of general L_p perturbations where certain pixels are fixed and the remaining pixels can be modified with an L_p perturbation, one way to accomplish this is to modify the conv_bound (https://github.com/IBM/CNN-Cert/blob/master/cnn_bounds_full_core.py#L199) and conv_bound_full (https://github.com/IBM/CNN-Cert/blob/master/cnn_bounds_full_core.py#L231) functions so that only weights corresponding to modifiable pixels are considered in the computation of the dualnorm variable.

Thanks, Akhilan

[EmanueleLM]

Thank you for the reply.

I've a doubt about the L__\inf norm bound: let's say that I'm interested in finding the lower bound's \epsilon by bounding just the first input of an image in a convolution (so those pixels at index [0,0,:], where the last layer is the input channel). Wouldn't it be enough just to change line 205 of conv_bound in cnn_full_bound_core.py, dualnorm = np.sum(np.abs(W[k,:,:,:])) to dualnorm = np.sum(np.abs(W[k,0,0,:])) That's because once I've calculated the upper and lower bounds in the first layer, by setting \epsilon to zero where required, the successive upper and lower bounds are calculated "recursively" starting from these results, as it is done from line 447 of function find_output_bounds. Does it make sense to you?

Thank you for your help.

Best.

[AkhilanB]

I believe in this case it is sufficient to change line 205 as you suggested, while also changing conv_bound_full to use a tensor \epsilon instead of a scalar \epsilon as done now.

To clarify the algorithm, let me refer to the lines from 413-437 in compute_bounds of cnn_bounds_full_core as "weight propagation" and lines 438-439 as "bound computation". For each layer (except the first), both steps happen to find the LB and UB bounds for that layer (for the first layer, bound computation can be done directly without any weight propagation as in line 444). As you point out, for the first layer the proposed modification to conv_bound would result in the correct bound computation for the first layer. Depending on whether the eps passed to conv_bound is a scalar or tensor, conv_bound will need to be further modified (when eps is a tensor, looping needs to be performed over more elements of W to compute UB, LB: for example, UB[:,:,k]+=eps[x,y]*np.sum(np.abs(W[k,x,y,:])) over x,y and UB[:,:,k] += mid).

Making only a subset of pixels perturb-able also changes the bound computation step of later layers. This is why conv_bound_full also needs to be modified. I think if conv_bound_full is correctly modified to use a tensor eps (a similar modification to conv_bound), then the bound computations will be correct.

[EmanueleLM]

Thank you again, you are really of help, anyway there's something I guess I'm missing.

In conv_bounds_full, in order to use a tensor for \epsilon, I have to specify to which indices we are acting (i.e. whose to deactivate will be put to zero), this makes sense to me. Let's suppose that I've calculated those bounds correctly and that they takes into account that some epsilon are zero, other's are not.

From the second layer on, when calculating the upper and lower bounds with conv_bound_full, the code at the final step will re-use the bounds calculated with conv_bounds, so why putting also here some epsilon tensors? I think that after the first layer, every layer should not put epsilon equal to zero because we already accounted for it in the first layer..

Am I missing something?

Thank you.

[EmanueleLM]

Ok I've seen that conv_bounds_full when invoked takes into account tensors A and B till the initial layer. Do you think that given as input a MNIST image (28,28,1) and an initial convolutional layer with 32 filters, this can be a workaround to try the results on the initial example. Modifying conv_bound_full, line 239 (for loop) from: dualnorm = np.sum(np.abs(A[a,b,c,:,:,:])) to: if A.shape == (1, 1, 32, 28, 28, 1): .... dualnorm = np.sum(np.abs(A[a,b,c,:,0,0]))

[AkhilanB]

Yes, for the bound computation at each layer, the results of the bounds computations at all previous layers are necessary including the \epsilon at the initial layer.

I think the solution you propose at line 239 should probably be: if A.shape == (1, 1, 32, 28, 28, 1): .... dualnorm = np.sum(np.abs(A[a,b,c,0,0,:]))

However, this only changes the dualnorm for the bound computation of layers with shape (1,1,32), not for all layers. Using this approach will probably require if statements for all different layers past the first one (for every layer past the first there will be an A tensor between that layer and the input of shape (layer.shape[0],layer.shape[1],layer.shape[2],28,28,1)). Alternatively, the if statement can be left out to change the bound computation for all layers.

Alternatively, I recommend modifying conv_bound_full to work with any tensor \epsilon. This way, there is no need to hard-code the perturb-able pixels in the dualnorm computation. Instead, \epsilon can be zero at the unperturbed locations.

For example, something like this might work:

@njit
def conv_bound_full(A, B, pad, stride, x0, eps, p_n):
    y0 = conv_full(A, x0, pad, stride)
    UB = np.zeros(y0.shape, dtype=np.float32)
    LB = np.zeros(y0.shape, dtype=np.float32)
    for a in range(y0.shape[0]):
        for b in range(y0.shape[1]):
            for c in range(y0.shape[2]):
                for x in range(eps.shape[0]):
                    for y in range(eps.shape[1]):
                        UB[a,b,c] += eps[x,y]*np.sum(np.abs(A[a,b,c,x,y,:]))
                mid = y0[a,b,c]+B[a,b,c]
                UB[a,b,c] += mid
                LB[a,b,c] += mid
    return LB, UB

[EmanueleLM]

That's absolutely great! I didn't notice that the dimensions of UB,LB in conv_bound_full were such that you could define an epsilon for the input so easily.

I think now I'm fine, if you could just wait few days then close the post so I can try some experiments.

Best, Emanuele

[EmanueleLM]

Now that the conv_bound_full is solved, I've done some experiments, but I've some concern about the results and how to apply the epsilon to conv_bound.

If I'm not wrong, conv_bound applies the epsilon to the weights that are interested by our verification: for example, if I want to consider just the perturbations on the first input (a (28,28) image), in a convolution with kernel of size (3,3), no padding and stride=1, in the dualnorm I use just dualnorm = np.sqrt(np.sum(W[k,0,0,:]**2)), while if I consider the first 3 input pixels (from the first row) I will set dualnorm = np.sqrt(np.sum(W[k,0,:3,:]**2)). With this rationale, if I'd like to modify the input at index (2,2), I will have to put dualnorm = np.sqrt(np.sum(W[k,:,:,:]**2)), because any weight in the first kernel of the convolution will be multiplied at a certain point to the specific pixel.

Is it correct? Because in this setting I find for example that the bound when I modify the first 3 pixels (which will be multiplied just by the first row of the kernel's weight) is bigger than modifying just the pixel at coordinates (2,2) (which will be multiplied, at different times, by all the kernel's weights).

Thank you.

[AkhilanB]

I believe in the case of only certain pixels being perturbed, conv_bound needs to be modified even further. Currently, conv_bound assumes that perturbations are the same uniformly over spatial locations, resulting in the output bound gap UB-LB being the same at all spatial locations. This approach does not account for spatially different perturbations. One approach may be to rewrite conv_bound so that it computes different bound gaps at different spatial locations like conv_bound_full. In fact, I think simply replacing conv_bound with the new conv_bound_full might work.

[EmanueleLM]

I've just eliminated the conv_bound part in find_output_bounds and substituted with this (where now the loop starts at 1 instead of 2), so now only conv_bound_full (the new version with tensors) is used. I got no error right now:

def find_output_bounds(weights, biases, shapes, pads, strides, x0, eps, p_n):
    #LB, UB = conv_bound(weights[0], biases[0], pads[0], strides[0], x0, eps, p_n)
    tmp = np.zeros(shape=(28,28)) 
    tmp[0,0] = eps
    eps = tmp
    LBs = [x0]
    UBs = [x0]
    LBs[0][0,0,:] -= eps
    UBs[0][0,0,:] += eps
    for i in range(1,len(weights)+1):
        #print('Layer ' + str(i))
        LB, _, _, UB = compute_bounds(tuple(weights), tuple(biases), shapes[i], i, x0, eps, p_n, tuple(strides), tuple(pads), tuple(LBs), tuple(UBs))
...

Do you think might work?

[AkhilanB]

If I understand the code correctly, now the eps passed to compute_bounds (and therefore also conv_bound_full) is a tensor, so I think the computations should be correct. The only thing I’m not sure is correct is the setting of the initial lower and upper bounds- instead of ‘LBs[0][0,0,:] -= eps’ the code might have to be something like ‘LBs[0] -= eps[:,:,np.newaxis]’. In any case, the final result should probably be the same since the final bounds don’t actually depend on the LB and UB at the input.

[EmanueleLM]

Amazing, I've some numerical error with some specific inputs on the UB,LB (they becomes nan), but apart from that, thank you very much, really appreciated your help.

Emanuele