search

MadryLab / robustness

A library for experimenting with, training and evaluating neural networks, with a focus on adversarial robustness.
MIT License
905 stars 181 forks source link

Clarification on epsilon values #31

Closed kiranchari closed 4 years ago

kiranchari commented 4 years ago

Hi, thanks for making this library!

I have a couple of questions about the epsilon budget -

  1. Are epsilons budgets for linf norms (such as 8/255) in pixel space ([0,1] and before normalization with mean/std) or model input space (pixel values divided by 255. and normalized by mean/std)? If they are in the latter, model input space, why are adv. samples clamped to [0,1] in both Linfstep.project() and L2Step.project()?

  2. Could you share the rationale for choices of epsilon (0, 0.5, 1., 2., 3) for l2 norm balls?

Thank you!

andrewilyas commented 4 years ago

  1. Epsilon is for images in [0, 1], normalization is wrapped into the model itself rather than the data augmentations.

  2. These values are pretty standard in the literature, and give reasonable accuracy values. In principle you can evaluate accuracy with respect to any epsilon.

kiranchari commented 4 years ago

Thanks for the clarification Andrew