Which of the provided images are unambiguous?

[davidwagner]

The contest provides three sets of images: train, test, and extras. Which of these have already been verified by taskers to meet the criteria to be a valid unambiguous image? It might be be useful to document this somewhere.

Looking through 40 images in extras chosen at random, it looks like these have not been filtered. Examples:

• extras/bird/aab056385368ffe0.jpg is truncated
• extras/bird/aabd131f8ea4d653.jpg is truncated and too small
• extras/bird/aac6db9a79526b08.jpg is truncated
• extras/bird/aaed9e10e854535b.jpg is egregiously truncated and too small (see below)
• extras/bird/aafd9b89de768256.jpg is too small
• extras/bicycle/aaa8f507a5a16f87.jpg seems likely to be too small because it is an upright bicycle viewed from the front, so the bounding box is very skinny
• extras/bicycle/aaa591e61575f966.jpg
• extras/bicycle/aaa792ed7e91d082.jpg seems likely to be too small because its bbox is skinny
• extras/bicycle/aaac8cd8a00c4ce6.jpg is truncated or obscured
• extras/bicycle/aaae5e8dd0264bf0.jpg is truncated
• extras/bicycle/aab94767160b07c6.jpg is truncated and seems likely to be too small
• extras/bicycle/aabeddab8c7195b8.jpg seems likely to be too small
• extras/bicycle/aabede57728825f0.jpg is egregiously truncated (see below)
• extras/bicycle/aac04786ce1990f7.jpg is truncated
• extras/bicycle/aacc8977cff9b795.jpg is truncated
• extras/bicycle/aad6b19bf5aae3d6.jpg seems likely to be too small
• extras/bicycle/aad6d1e25965a51d.jpg seems likely to be too small
• extras/bicycle/aada107f62a8e618.jpg seems likely to be too small, and truncated
• extras/bicycle/aadabb937c715800.jpg is egregiously not a bicycle (see below)
• extras/bicycle/aadfa07ce43f0d2a.jpg is egregiously truncated
• extras/bicycle/aae7b07726e5c5ee.jpg is truncated
• extras/bicycle/aaf39239027d3b06.jpg is truncated
• extras/bicycle/aaf7891202471a5d.jpg is truncated
• extras/bicycle/aafbd4ab667d5680.jpg seems likely to be too small
• extras/bicycle/aaff6738c4351441.jpg is truncated

[carlini]

Yeah, so extra/*/* is an unfiltered dataset that will have a lot of noise but it's designed to have as images as possible so that we can train useful classifiers. (For example, if you use an imagenet classifier as a feature extractor, then a single linear projection layer can get 99% test accuracy by training on extra.)

I believe (in theory) that train/*/* and test/*/* should satisfy the constraints. We're going to (very shortly -- early next week?) release a new version of the dataset that will rename current test to val, and then introduce a new test dataset with in-the-wild images that will be a subset of the held-out test set to be used in the full contest.