comments .. labelme mask tool

[dobkeratops]

not really an issue, just a talking point.. been messing with the LabelMe mask tool, which fills out from scribbles to give per-pixel labelling ;

This is actually rather awesome. I hadn't bothered using it before because I thought it would be too fiddly with the trackpad, but it's still better than I thought it would be.

(I wonder what it's like with a touchscreen)

They let you make one 'mask' at a time, assigning a label to it;

suggestion 1: - left/right mouse buttons to make posative/negative scribbles without needing any switching (although they probably didn't bother because of the number of non-mouse users)

suggestion 2

• it would be nice to have a combined mode where you can make several mutually-exclusive per-pixel labels by switching the current 'scribble colour' from a palette (e.g. street scene: pavement,cars,people,sky,buildings,road) . Where you have ambiguity ('fine foliage over buildings') you'd need a distinct 'colour' for that.

Nonetheless, they have the right priorities giving you the 'single mask' mode first because it's more versatile, e.g. you could go in and overlay materials on buildings ('bricks , stone' etc), and of course the ambiguous case of overlap (foilage where you can see trees and buildings through it)

I might be curious to make a tool to do this locally (i.e. just use a paint program to do the multicolour scribbles in a layer.. and submit the masks in their format? .. I wonder if there's already something to do that)

[bbernhard]

I finally managed to create a LabelMe account and played a little bit with it today. As you already mentioned, there are some parts that could really be improved. It took quite some time for me to get used to the annotation tool. Sometimes I was really frustrated, because I wanted to do a certain action (e.q close a popup without entering something, because I want to do some refinements), but the tool didn't let me.

But you are right, the scribbling mode works pretty well. I also wanted to try it on a smartphone, but it looks like that LabelMe annotation tool isn't working on a smartphone at all (at least It didn't work on my smartphone).

I really like your ideas - those would make it even more awesome.

I might be curious to make a tool to do this locally (i.e. just use a paint program to do the multicolour scribbles in a layer.. and submit the masks in their format? .. I wonder if there's already something to do that)

Does LabelMe offer a possibility to submit something (expect images)? I am still exploring it's capabilities, but I currently only found the possibility to download data.

[dobkeratops]

I'm assuming there must be some protocol to communicate with the server, so I imagine it should be possible to inspect the source code and figure out how to submit masks. I guess it must submit the scribble paths , and calculate on the server; The idea would be, given N mutually exclusive labels, to submit each in turn , but with the sum of the others as the 'negative'.

of course the real benefit of their cloud approach is anyone, anywhere can contribute labels to the same dataset.. you can do something 'half arsed' (spend a short time putting a couple of labels on one image and move on quickly), and there's still a chance someone else can come along and improve (i.e. adding more labels or parts)

but the tool didn't let me.

A few more hotkeys would help them. It's not terrible though, e.g. it does have the ability to enter one then you use 'add parts' to annotate pieces, 'stop parts' when you've done the last one. But there's so many other shortcuts you could add . The ability to split and move points afterward would be really helpful, e.g. blocking out then go back and refine.. (and the ability to share boundaries.. you often have to trace the skyline twice etc)

I think it's great that it's got both the outline and mask approaches. I can't say if one or the other is superior.

[dobkeratops]

I currently only found the possibility to download data.

i think you can submit image collections (as public or private datasets) but I haven't seen an official way to submit label/mask data.

[bbernhard]

i think you can submit image collections (as public or private datasets) but I haven't seen an official way to submit label/mask data.

I see. But you are totally right, it would be really cool if submitting masks would be possible. :)

A few more hotkeys would help them.

Definitely. Another possible improvement would be to have some sort of (interactive) tutorial on how to use the tool. As LabelMe is targeting people with technical background it's probably not that important, but for establishing a broader user group, it could be helpful. At least for me it wasn't always clear how to accomplish certain things. Sometimes I thought: "Hmm...that's not working." until I figured out, that I was just using it the wrong way.