example of using ExpectationOverTransformation (EoT) wrapper with preprocessing defence

[Embeddave]

Is your feature request related to a problem? Please describe. Hey again art devs, this is a follow up to #702 and some discussion on #209

We're trying to use the EoT wrapper to replicate a result from the Athalye et al. 2018 "Obfuscated gradients" paper, where they report using EoT to successfully attack a classifier that uses a Total Variance Minimization preprocessing defence.

Is there an example in the docs somewhere of using the EoT wrapper with an actual preprocessing defence? The closest thing we could find is this unit test:
https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/62ffe7c951d8a60d49a9ea6ac7b04aa4432a3fb7/tests/wrappers/test_expectation.py#L52

I'll paste the whole thing here for context, basically it shows the simplest possible example of defining an iterator to give to an eot instance as the transformation argument:

        # Build KerasClassifier
        krc = get_image_classifier_kr()

        # Get MNIST
        (_, _), (x_test, y_test) = self.mnist

        # First attack (without EoT):
        fgsm = FastGradientMethod(estimator=krc, targeted=True)
        params = {"y": random_targets(y_test, krc.nb_classes)}
        x_test_adv = fgsm.generate(x_test, **params)

        # Second attack (with EoT):
        def t(x):
            return x

        def transformation():
            while True:
                yield t

        eot = ExpectationOverTransformations(classifier=krc, sample_size=1, transformation=transformation)
        fgsm_with_eot = FastGradientMethod(estimator=eot, targeted=True)
        x_test_adv_with_eot = fgsm_with_eot.generate(x_test, **params)

Describe the solution you'd like If there's not already a concrete example in the docs of using eot with a preprocessing defence, could one be added?

Describe alternatives you've considered Here's an attempt I wrote out before: https://github.com/Trusted-AI/adversarial-robustness-toolbox/issues/289#issuecomment-683122165

Basically our guess as to the correct way to do this looks something like:

tv_preprocessor = TVMinimization()

def transformation():
    yield tv_preprocessor

# clf_art is a PytorchEstimator instance, BATCH_SIZE is some constant
eot = ExpectationOverTransformations(classifier=clf_art, sample_size=1, transformation=transformation)
cw_eot = CarliniL2Method(classifier=eot, batch_size=BATCH_SIZE)

It's not at all clear to us whether we are properly implementing the iterator that the eot wrapper expects to receive as a transformation.

What is the logic for using an iterator here instead of just letting the user pass an instance of a preprocessing defence? Is it to make ExpectationOverTransformation a more general wrapper? Some context on this in docs would help, maybe with that concrete example in the docstring

I admit I'm not a good enough Python programmer to grok what's going on. But if that's true for me I think it might be true for other researchers trying to leverage the abstractions in art also.

Our other alternative would be ... use the armory approach? https://github.com/kevinmerchant/armory-example/blob/adaptive_tutorial/tutorial/patch_loss_gradient.md But we'd prefer to not have to special-case anything and use "pure art" where we can

Additional context n/a

[Embeddave]

update: when we try the naive approach outlined above, we get this error:

TypeError                                 Traceback (most recent call last)
<ipython-input-26-2a26c343c473> in <module>
      1 x_at_wout_TV = attack.generate(x, y)
      2 
----> 3 x_at_w_TV = attack_eot.generate(x, y)

~/anaconda3/envs/gard-owlre/lib/python3.7/site-packages/art/attacks/attack.py in replacement_function(self, *args, **kwargs)
     72                 if len(args) > 0:
     73                     args = tuple(lst)
---> 74                 return fdict[func_name](self, *args, **kwargs)
     75 
     76             replacement_function.__doc__ = fdict[func_name].__doc__

~/anaconda3/envs/gard-owlre/lib/python3.7/site-packages/art/attacks/evasion/carlini.py in generate(self, x, y, **kwargs)
    278                 x_adv_batch_tanh = x_batch_tanh.copy()
    279 
--> 280                 z_logits, l2dist, loss = self._loss(x_batch, x_adv_batch, y_batch, c_current)
    281                 attack_success = loss - l2dist <= 0
    282                 overall_attack_success = attack_success

~/anaconda3/envs/gard-owlre/lib/python3.7/site-packages/art/attacks/evasion/carlini.py in _loss(self, x, x_adv, target, c_weight)
    148         l2dist = np.sum(np.square(x - x_adv).reshape(x.shape[0], -1), axis=1)
    149         z_predicted = self.estimator.predict(
--> 150             np.array(x_adv, dtype=ART_NUMPY_DTYPE), logits=True, batch_size=self.batch_size,
    151         )
    152         z_target = np.sum(z_predicted * target, axis=1)

~/anaconda3/envs/gard-owlre/lib/python3.7/site-packages/art/estimators/classification/classifier.py in replacement_function(self, *args, **kwargs)
     69                 if len(args) > 0:
     70                     args = tuple(lst)
---> 71                 return fdict[func_name](self, *args, **kwargs)
     72 
     73             replacement_function.__doc__ = fdict[func_name].__doc__

~/anaconda3/envs/gard-owlre/lib/python3.7/site-packages/art/wrappers/expectation.py in predict(self, x, batch_size, **kwargs)
     68         """
     69         logger.info("Applying expectation over transformations.")
---> 70         prediction = self._predict(next(self.transformation())(x), **{"batch_size": batch_size})
     71         for _ in range(self.sample_size - 1):
     72             prediction += self._predict(next(self.transformation())(x), **{"batch_size": batch_size})

~/anaconda3/envs/gard-owlre/lib/python3.7/site-packages/art/estimators/classification/classifier.py in replacement_function(self, *args, **kwargs)
     69                 if len(args) > 0:
     70                     args = tuple(lst)
---> 71                 return fdict[func_name](self, *args, **kwargs)
     72 
     73             replacement_function.__doc__ = fdict[func_name].__doc__

~/anaconda3/envs/gard-owlre/lib/python3.7/site-packages/art/estimators/classification/pytorch.py in predict(self, x, batch_size, **kwargs)
    187 
    188         # Apply preprocessing
--> 189         x_preprocessed, _ = self._apply_preprocessing(x, y=None, fit=False)
    190 
    191         # Run prediction with batch processing

~/anaconda3/envs/gard-owlre/lib/python3.7/site-packages/art/estimators/estimator.py in _apply_preprocessing(self, x, y, fit)
    238         # y = check_and_transform_label_format(y, self.nb_classes)
    239         x_preprocessed, y_preprocessed = self._apply_preprocessing_defences(x, y, fit=fit)
--> 240         x_preprocessed = self._apply_preprocessing_standardisation(x_preprocessed)
    241         return x_preprocessed, y_preprocessed
    242 

~/anaconda3/envs/gard-owlre/lib/python3.7/site-packages/art/estimators/estimator.py in _apply_preprocessing_standardisation(self, x)
    290                 div = np.asarray(div, dtype=x.dtype)
    291 
--> 292             res = x - sub
    293             res = res / div
    294 

TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

[beat-buesser]

Hi @Embeddave Thank you very much for raising this issue, we'll try to take a look at it as soon as possible.

[Embeddave]

Thank you @beat-buesser

Is there any way you or @mathsinn could give us just a very brief example here on this issue of the correct usage of the eot wrapper when combining it with a real defense? E.g., TotalVarMin?

It would be great to get this documented in the long run, and I could possibly contribute a PR if you are interested, but right now we just need one really quick example of the right way so we can move past a research roadblock.

[Embeddave]

For the example above @kevinmerchant pointed out we were missing a while True statement (as in the unit test)

Our example now runs without that error (🙏 Kevin)

[beat-buesser]

Hi @Embeddave Sorry for the delayed response, but that's great to hear! Does this solve all your issues or is there anything else we could help with?

[Embeddave]

Thank you @beat-buesser

Can you let us know if the while True approach is the "right" way to use ExpectationOverTransformations with an art preprocessing defense? i.e., should we just always be defining an "infinite iterator" that simply returns our defense instance?

Still trying to understand the need for requiring transformation to be an iterator. I guess it's to give the the user more flexibility in what transforms get applied each time that iterator.next gets called? E.g., I could define a transformation that randomly picks one of some set of preprocessing defenses each time it's called

[beat-buesser]

Hi @Embeddave I'm not completely sure about art.wrappers.expectation.ExpectationOverTransformations but based on its implementation it averages calls to the classifier's methods loss_gradient or predict over sample_size samples and the transformation iterator should return a different transformed sample on each call. Based on this I don't think a while True is required in the transformer. Since ART 1.5 we have now also full support for Expectation over Transformation with framework-specific implementations with gradient backpropagation through EoT in the new module art.preprocessing.expectation_over_transformation with a first example for image rotation for classification.

[Embeddave]

Good to know, thank you @beat-buesser

I will close this and we'll look at the EoT class in preprocessing