openphilanthropy / unrestricted-adversarial-examples

Contest Proposal and infrastructure for the Unrestricted Adversarial Examples Challenge
Apache License 2.0
327 stars 62 forks source link

Add "common corruption" transformations as a warmup attack #40

Closed nottombrown closed 5 years ago

nottombrown commented 5 years ago

I think that the transformations from Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations by Dan Hendrycks and Tom Dietterich would be a good addition to the warmup attacks.

image

Three reasons why I think that this would be a good addition:

  1. Their corruptions clearly illustrate a set of non-lp-restricted attacks
  2. These corruptions are strictly easier than an unrestricted adversary
  3. They have a simple and well-designed codebase

Tasks

nottombrown commented 5 years ago

I think that this would be most relevant for the bird-or-bicycle dataset (because that's what the original authors designed it for), and need not be added to MNIST.

nottombrown commented 5 years ago

@hendrycks has volunteered to be the author of this after the ICLR crunch https://github.com/hendrycks/robustness/issues/2#issuecomment-423273663 Thanks!

nottombrown commented 5 years ago

Added in #51