'scene labels' vs 'object labels'

[dobkeratops]

this is my highest priority suggetion: At the moment the site presents you with a very limited set of labels; as such the range of images you can submit is very restrictive.

How about adding some very broad categories for labels applying to entire scenes; these would not be suitable for area tagging, but you'd later go in and crop regions to distinguish the contained objects. This would instantly broaden the contributions to the database.

Scene labels still have training value, since different scenes contain different correlations of objects. , and it's possible to divide object recognition into multiple networks, e.g. the result of a 'broad classification' network selects one of several domain-specific 'narrow-classification networks' to use to refine. (e.g. you could have 4 networks.. 1='urban v domestic v nature', 2.1='urban objects', 2.2='domestic objects', '2.3=nature objects', and based on the result of '1', you then feed the image into 2.1,2.2, or 2.2)

Suggested Scene Labels:

**domestic**   : interior domestic household scenes, containing everyday objects
 **kitchen**, **bathroom**, **bedroom**, **living room** 
   : subtypes of 'domestic' , distinguishable by different objects/furniture
**urban**   : street scenes - houses, cars, pedestrians etc.
 **suburban**, **city**, **town centre** (emphasis on different styles of building)
**architecture**      :e.g. photos focussing on specific buildings, interiors of churches/stately homes etc
**industrial**  : factories, power plants, workshops (training robots to distinguish out components..)
 **workshop** scenes containing tools, components (garage, maker)
**office**  : workplaces (interiors, not domestic or industrial)
**rural**    : bit more specific than 'nature'.. villages,etc, but not quite 'agriculture'.
**agriculture**   : farm settings; suitable for task-specific labels (agribot training..)
 **livestock** slightly more specific than agriculture.. pictures of cows/sheep etc.
 **crops** more specific than 'agriculture', i,e. *not* livestock.
**foodstuffs** ? (general pictures of fruit bowls etc?)
**nature**      : forests ,wildlife etc.
**garden**   : (somewhere between 'domestic, urban, nature, agriculture', think also of allotments; again agribot training)
**geology** : caves, rock formations, cliffs,canyons etc?
 **crowd ** : busy streets
 **social ** : pictures of gatherings 

very broad catch all categories
**plants** 
**animals**
**machines**
**vehicles**
**people** 

[dobkeratops]

example - this is the kind of thing that springs to mind when I hear of a labelling application.. get the most value out of every image by labelling absolutely everything, and encourage users to get more precise. If you just uploaded this as an image of a 'building', later people could have come along and labelled the cars. And so on.. (this is why I found the initial narrow label, and single label per image rather too restrictive). Leave things open ended, because later users may know more. One person has a camera in an interesting place.. another is sat at home with time on their hands to do labelling

[dobkeratops]

Just thought about this again when going through the site to do some image donation.. (the script is great but the site UI is what most people will use)

I'd suggest scene labels to give the most useful single label for an entire image; it would make sorting & searching easier too.

These would have overlap but you can chose on a case by case basis.. what single word best describes a particular scene? That would help whole image training?

existing scene labels kitchen forest beach

new suggestions - from memory in the dataset.. street residential area landscape town rural area urban area city city centre town centre village park garden office home shop living room restaurant cafe shopping mall workshop garage jungle valley industrial area construction site canal (canal is actually similar to river but it's an artficial channel) harbour classroom library museum hospital farm highway (specialisation of road, often means it's going through landscape) port

.. each scene label could guide the 'popular labels' in the 'add labels' view.. eg 'street' would suggest labels: road, pavement,tree,building,house,car,person,bin; 'park' would suggest labels foliage, tree trunk, bush,grass,path,wall. 'farm' would suggest fence, field,tractor,barn, ... and so on. You could even add those all as default tasks, and use the 'skip' to remove a specific label

[bbernhard]

Thanks for the labels list - those labels definitely make sense!

As you've mentioned the donate page: We've talked about the donate page a while ago and you suggested to make the < unlabeled > label the default one. I recently thought about that again and I am thinking whether we should go even a step further and remove the labels list completely from the donate page?

In my opinion the donate page is only useful in case you wanna donate a handful of images. If you wanna donate more than that, it's probably better to use a scripted approach (maybe we should cleanup the script a bit, create a commandline tool out of it, write a better documentation of the tool and add a reference to the donate page?).

I think with the current approach it's pretty easy to label an image wrongly (in case you forget to change the label in the label dropdown before uploading). I remember when I first posted about ImageMonkey at reddit (~1-2 years ago) there were quite a few donations from people who wanted to give ImageMonkey a shot. However, a significant portion of those images were obviously wrong labeled (tree vs car, etc) after upload. But to be fair: This was still the time where a new random label was chosen from the label dropdown after every upload...so maybe that was the reason for that. :D

Another problem I see is, when the user uploads a completely different set of images where we do not already have labels defined for that. In the best scenario, the user just selects < unlabeled >, in the worst scenario he could pick a label that's similar enough but not correct.

What do you think about that?