discussion.. alternative to 'pro mode' , upload with painted annotations?

[dobkeratops]

something that might be reasonably easy without a load of UI.. imagine just loading the images into a paint program, and using thin scribbles in primary colours for annotations (e.g. 0xff0000 0xffff00 0x00ff00 0x00ffff 0xff00ff 0x0000ff .. those wouldn't appear in natural images; filter these pixels out, replace with averages of surroundings, and use those as labels. you could also use some judgement based on gradients to distinguish from natural images. They must be drawn without anti-aliasing)

you could provide a ui for mapping the colours to labels, or even use your other modes to confirm what they are ('is this a ...')

beyond that you could consider separate images (like photoshop layers) holding the annotations.

I wonder if there's already photoshop/gimp plugins to facilitate that (e.g. imagine something that works a bit like the LabelMe scribble tool).. seems like an obvious idea. those packages even have a concept of channels, so you could have arbitrary masks encoded; but the idea of scribbling directly in the image could keep things in one place. If worried about losing pixels under the scribbles, perhaps you could just upload both the annotated and un-annotated versions. I think most natural photos are a bit blurry anyway

I imagine something like this wouldn't get as much use as something 100% browser based, but it could be a stopgap that helps you expand the dataset in a different way.

of course the magic of the browser-based approach is you know others can build on your contributions, you don't have to install anything, you get direct interactive feedback. This would be more of a batch-mode tool for people who have taken the time to read some instructions etc..

Another way to do this might be to rely on an offline tool for a conversion of locally scribbled images into 'LabelMe' format (although anything a user needs to install decreases the chance of use)

[bbernhard]

Interesting idea!

However, I am struggling a bit to see the user group that would use that?

I do not have much experience with Photoshop/Gimp but I imagine it's pretty cumbersome to open each image individually, do some scribbles, save them and them upload them? Or do you have another use case in mind?

I very much like your idea of "outsourcing" the work to other tools and just push back the results. I have no idea if the author of LabelImg[1] is open to that, but I could imagine a "opt-in" possibility which pushes annotated data back to ImageMonkey. (if not we could always create a fork)

[1] https://github.com/tzutalin/labelImg

[dobkeratops]

but I imagine it's pretty cumbersome to open each image individually, do some scribbles, save them and them upload them?

indeed, it wouldn't be as user-friendly has having the workflow integrated in a website; there's a lot to communicate to users (whereas the website can give you feedback about specifics); however, you'd spend a relatively long time on each image, which would give a lot of training value, because it's per-pixel labels; and you could still show examples, and show what the per-pixel segmentation looks like.. and put them through the same human validation passes as other contributions.

it'll be one 'drag-drop' to upload image selection, but you might do 10-100 'brush strokes' within each. regarding 'opening each image individually', it's down to the OS to be good at scrolling through your hard drive .. many file-browsers display thumbnails, you can associate applications with file types so double-clicking opens an image directly into your preferred image editor .. etc.

[bbernhard]

it's down to the OS to be good at scrolling through your hard drive .. many file-browsers display thumbnails, you can associate applications with file types so double-clicking opens an image directly into your preferred image editor .. etc.

good point :)

Just out of interest: Have tried creating some scribbles with a normal graphics program (Photoshop, GIMP...etc)? I am not very good with those programs, so I am really curious how that turned out and if you were satisfied with the workflow/result. :)

[dobkeratops]

Have tried creating some scribbles with a normal graphics program yes; this demonstrates the idea of marking multiple mutually exclusive labels (i.e. you'd fill out from each scribble simultaneously to make non-overlapping masks) this is in 256colours so the original image is messed up a little.. (the package I was using didn't have an option to even dither it on converting). I imagine you'd have to upload the unscribbled original aswell. The suggestion I made would require a lossless format (i.e so the scribble colours remain pure).

so yeah.. there's fiddlyness to deal with, fewer people would bother or get it right .. then again it seems the world would have many potential uses for image segmentation so i'm sure there's things out there to do it (I still haven't looked)