Review of existing datasets

[davanstrien]

https://paperswithcode.com/datasets?mod=images&task=face-recognition

[davanstrien]

Some notes on COCO:

• 80 total categories labelled in total list
• Each coco label is a subclass of a 'super category' mapping

A subset of labels which sit under 'person','animal', 'vehicle':

subset = [cat['name'] for cat in cats if cat['supercategory'] in ['person','animal', 'vehicle']]
[u'person',
  u'bicycle',
  u'car',
  u'motorcycle',
  u'airplane',
  u'bus',
  u'train',
  u'truck',
  u'boat',
  u'bird',
  u'cat',
  u'dog',
  u'horse',
  u'sheep',
  u'cow',
  u'elephant',
  u'bear',
  u'zebra',
  u'giraffe'],

• Potentially annotate at the supercategory level and then evaluation can be done for any of the lower levels that fit into that super-category?

• This way we don't actually evaluate accuracy on that lower level so if a model predicts zebra it still counts as a correct prediction because it is an animal.

• Alternatively we could do all of these categories but this might be more challenging to set up as an annotation task in a clean way?

• One other challenge is that we might ask people to annotate 'animal' and they annotate an animal that doesn't appear in the subcategories? i.e. they annotate a snake that isn't captured in the coco annotations.