search

ImageMonkey / imagemonkey-core

ImageMonkey is an attempt to create a free, public open source image dataset.
https://imagemonkey.io
47 stars 10 forks source link

experimenting with UI ideas #49

Open dobkeratops opened 6 years ago

dobkeratops commented 6 years ago

https://dobkeratops.github.io/imageannotation.html

might be interested, I've tried out some ideas:

Not sure how far I'll take this though.. the amount of UI work does explode, there's little rough edges all over the place. It doesn't connect to anything yet.

I still think your tool with bounding rects is a great path to continue with (hierarchical ellipses would work very well).

bbernhard commented 6 years ago

Really cool stuff!

I think that could be something we could build upon. If the individual UI elements are separatable and individually usable we could use them at different places. e.q: in the pro mode we could expose the whole annotation tools, whereas other screens/tabs only contain a subset.

What we should also consider is: mobile friendliness/responsive design. I think most "hardcore annotators" probably won't use a smartphone/tablet, but I think for the occasional annotator a mobile friendly design would be awesome. That's one of the things that bugs me the most about LabelMe (apart from the fact that they aren't using SSL): it's not mobile friendly at all.

I am usually the type of guy that stumbles across blogs and new technologies when I am on the way to my workplace. When I am sitting in the tram, I have the time to read blog articles, reddit and HackerNews. If I now stumble across something interesting I immediatelly want to try it out, see what I can do with it and play a little bit with it. If it's a site that doesn't work on mobile I usually end up browsing away again (even if it's something that sounds super cool) without setting a bookmark.

Another completely different approach to minimize the UI work: Can we maybe use existing (desktop) applications and swap/extend the backend, so that it operates on the ImageMonkey dataset instead? In that case we would only need to adapt the ImageMonkey REST API so that it accepts other type of annotations (point/edge, lines..). The complicated drawing part will then be handled by an existing application.

I wonder if there already exists an open source application with great annotation functionalities that we could build upon?

[1] https://github.com/wkentaro/labelme [2] https://github.com/tzutalin/labelImg

dobkeratops commented 6 years ago

"Can we maybe use existing (desktop) applications and swap/extend the backend, "

seems like a good idea, though I tend to think the web-based approach is most interesting because people dont have to bother installing anything.. 'LabelMe' remains the most interesting one out there. Having said that a desktop app can be slicker.. (as a native code person, I hold a ray of hope that webassembly might blur the lines between desktop and web more)

, but I think for the occasional annotator a mobile friendly design would be awesome.

Agree. the touchscreen does need a different approach. 'touch the squares' .. enhanced with quad-tree like refinement perhaps? (the system could detect areas with many changes and present them to the next user zoomed in, with enhanced resolution..)

bbernhard commented 6 years ago

seems like a good idea, though I tend to think the web-based approach is most interesting because people dont have to bother installing anything.. 'LabelMe' remains the most interesting one out there. Having said that a desktop app can be slicker.. (as a native code person, I hold a ray of hope that webassembly might blur the lines between desktop and web more)

totally agree on that one. I also think that a web-based approach has it's advantages over a desktop application. I just thought, that maybe there is a really cool annotation tool out there, which we could easily extend with ImageMonkey support, while we are building the web based approach. I would see that more as a intermediate solution to provide people a full blown annotation tool while we are extending the web base annotation tools.

But up to now I haven't found anything that provides more than simple bounding rectangles for annotating.

As you mentioned WebAssembly: There is also qmlweb [1] which aims to bring the power of QML to the web browser. But the last time I tried it, it was pretty slow :/

Agree. the touchscreen does need a different approach. 'touch the squares' .. enhanced with quad-tree like refinement perhaps? (the system could detect areas with many changes and present them to the next user zoomed in, with enhanced resolution..)

Really like that idea :)

[1] https://github.com/qmlweb/qmlweb

dobkeratops commented 6 years ago

At this point for personal use, I have enough in my tool to prefer using it over LabelMe: the key thing is the subdivide/edit tool (block it out, refine it). This does make me want to figure out how to connect it properly to something for storage. (EDIT,I've changed it now to download a file and you can drag it back in .... still not enough for casual users IMO as you have to manage the files yourself).

If your DB stored polys in addition to the rotated bounding-boxes, perhaps I could have a go at pushing to that? (again I could just write out LabelMe format, much of a muchness but XML instead of JSON)