openphilanthropy / unrestricted-adversarial-examples

Contest Proposal and infrastructure for the Unrestricted Adversarial Examples Challenge
Apache License 2.0
327 stars 62 forks source link

What's the definition of occluded? #83

Open davidwagner opened 3 years ago

davidwagner commented 3 years ago

I'm wondering what counts as 'occluded'. Are these images of bicycles occluded?

train/bicycle/08495fcbbb5651d7.jpg - is it occluded by the light pole in front?

image

train/bicycle/60d1dbbc4fb92378.jpg - is it occluded by the rider's right leg and/or right arm?

image

Is a bicycle with a rider on it almost always going to be considered occluded?

carlini commented 3 years ago

Okay yes, this is a case I don't think we thought carefully enough about. There are some other images that are like this (e.g., birds in cages or behind a fence). I think it's probably a clearer task to disallow these images (otherwise: how much occlusion is okay? is a fence? a poll? a fire hydrant? at some point it's too much), but as you say that makes especially the bicycle class with a person it it difficult...

Do you have any thoughts?