dobkeratops
commented
6 years ago heres another experiment in GIMP, different workflow idea.. (i) draw major boundaries as black outlines in a layer (ii) flood-fill between them with color coding
there's no overlap and it's limited to how ever many colours you can easily distinguish, e.g 7 ish (unless you split into more layers, in which case you might aswell go back to channels)
it's easy to draw the boundaries, but you occasionally miss parts you can set up the layer opacity to see what you're doing it is easier to flick between a few categories in the colour palette.. there's less commands to use
plus points.. easy use of paint tools/tablet, the work done for each edge is shared between adjacent regions, the data is just a second image. minus - can only place limited labels with colour coding, and can't describe occluded parts.
bbernhard
Basically annotation is a painting task.
The recent tweaks have made many cases much easier (hence see the improved activity chart :) ),
I can explain clearer now what was wrong originally, and where to go to improve further:-
the original workflow is like being told to paint images one colour at a time "here's a red brush.. here's a random image.. draw all the red. now here's a green brush.. here's a different random image.. draw all the green ..etc"
Now with the browser mode, at least you pick one 'brush' and find the images where it's ok ... hence success with road, pavement (clear areas) - and of course the original single-subject stock photos were probably fine too.
The "add labels" mode works ok because it's more like asking "what palette does this image need?" .. that's often doable.
But the best way to handle an image is to see the context and gradually refine it.. (i) pick the predominant areas, block them out ("start with the mid-tones and broad brush shapes"), then refine by clipping details out.(add the highlights and shadows.. refine the boundaries..). the optimum order depends on the size and specific shapes (zoomed in, you clip out what isn't the subject.. but zoomed out, you start with the background then draw the subjects)
Saying "the unpainted area is a certain set of labels" might work ok, because this is like a background fill. (see also the idea of an 'unoccluded sky' label, and 'anti-label'- draw all the 'not-tree' in a forest). imagine being able to plan the task for an image, e.g. "we will start by filling with {this mix..}, then paint over {these details..}". (the idea of saying "un-annotated pixels are a mix of the unused labels" is kind of like that but more dynamic). I think that's also where I was going with the requests for "sparse/dense, few/many, foreground/background" label qualifiers: Those would hint whats the background fill, and what tasks are optimum to do first (related idea in rendering.. https://en.wikipedia.org/wiki/Painter%27s_algorithm, a layering order refines the boundaries). see #149 - the need for arbitrary expansion of the label list: drawing any object for occlusion is useful information.. waiting for labels is like saying "we can only draw certain pieces". it's often easier to draw the occluder rather than the precise boundary of a part you can see. e.g here's a peice of pavement/building, but to precisely identify the pixels today i'd have to trace around the litter bin, because there's no litter bin label yet.
. It's fewer clicks to draw the broad area for the pavement & building, then the occluding object infront... (more occluding 'containers'. imagine being able to draw those in.. rather than tracing the outline around them
)
Going back to the label/color analogy, the label list needs to be exhaustive - imagine trying to paint without all the primary colours.. (e.g. if there's no label "litter bin", a broader label "container" could still cover it)
(see also per-label occlusion hints, e.g. it's probably ok to assume certain labels(ground-types) occlude others (moveable object types), but over-ride might be useful)
Telling what colours are can be difficult in the real world because of relative perception.. labels aren't far off, because many shapes in the distance aren't clear, and there's overlapping categories.. but sometimes once you figure out surrounding context, those difficult parts get easier.
From this POV i'll re-iterate the idea of general label blending #141: you do get areas where it's optimum to mix '2 colours' (2 labels).. this is especially true of any organic parts (e.g. mixture of soil, bushes, grass, fallen leaves,bark,moss) but also anything moving. You sometimes get vegetation growing between paving stones (so a label "pavement/grass" would be optimal). see also #142 #84
Then there's bits of road covered in gravel around a construction site ("gravel/road"), moving crowds, trash or litter (cleaning bot..)
Here's a nice example of complex ground:-
you might start with a blend 'concrete/soil/grass/gravel' to fill the floor,("fill with a grey/green/brown random texture") then clip out the areas which are clearly one or two types ('the clear patches of grass, clear patches of concrete, etc'). as I'm looking at it now i'm still finding it hard to guess what the distant areas are (dry soil, or gravel, or dirty concrete?). It looks mostly concrete.. but the presence of grass implies other parts must be soil. regarding distant vegetation, you might say 'grass/foilage' (for the bushier parts)