Incorrect labels in training set?

openphilanthropy / unrestricted-adversarial-examples

Contest Proposal and infrastructure for the Unrestricted Adversarial Examples Challenge

Apache License 2.0

327 stars 62 forks source link

Incorrect labels in training set? #82

Open davidwagner opened 3 years ago

davidwagner commented 3 years ago

These look incorrectly labelled to me:

train/bicycle/17de8639a4d96c37.jpg (less than 25% of the area of the image):

train/bicycle/59b8c06908041df6.jpg (less than 25% of the area of the image):

train/bicycle/8c300972d999d02c.jpg (it's a tricycle, not a bicycle):

train/bicycle/cf71e106f19e6c23.jpg (each bicycle has a bounding box that is tall and skinny and hence less than 25% of the area of the image):

train/bicycle/eeb891e2c9316449.jpg (less than 25% of the area of the image):

carlini commented 3 years ago

You're definitely more discerning than the reviewers! The tricycle is especially wrong.

I think (@nottombrown correct me if I'm wrong) the reason that these don't satisfy the 25% rule is that we added that requirement after collecting the dataset used for the train set. We should correct this, though.