suggestion - label palette (eg for humans)

[dobkeratops]

imagine being able to toggle the label list with some theme specific overlays, the most obvious one being person parts; (no change to the label data, just a way to visualise and select) id imagine bitmaps with colour coding sych that you could click on the reference icon/image to get the part,

here’s a mickup of what i’d imagine the person part icon would look like..

a similar thing could be done for street scenes (road,curb,pavement etc), and vehicle parts

could this be user defineable? ive found how to run a test server and serve pages on my network (and i have some hosting, but no domain name) perhaps you could the user specify a URL for some JSON defining these schemes; ive also seen how a page can pass values via “local storage” (perhaps i could make a page to set things up that the main ui oicks up on).. but i suspect the json is best.

my knowledge of web ui is very hazy.. i’d probably struggle to fork your UI myself, although its not out of the question..

benefits:

• groups commonly co-occuring labels
• define a scheme for a particular training task (w examples here are human pose and street scenes)
• easier to use with the ipad £ pen ( eg painting the mockup was easy)
• the reference images can guide how to use them
• makes validation easier (check the colour coding, could cycle the colours eg left/right, libs groupings and check its all consustent)
• for me, i actually have to stop and think how to get left & rught correct - and i do find i made mistakes (i later correct with rework). copyiing from a reference , it’ll be easier.. just match visually “this bit is that bit” instead of having to thrink through “verbally”

[dobkeratops]

mockup: how it could be used in the editor’

• json description sets it up (eg name of the scheme, filename of bitmap, label colour codes and names),
• a selector at the top lets you toggle between the standard label list and a registered template
• clicking on the. template image picks a label by its colour code
• colour coding used in the image?
• current label name showed at the botton as normal, can still manually enter a label name here?
• maybe in the template specify a greying alpha factor for the image to let colout coding stand out

[dobkeratops]

example 2: street scene. in this example ive also greyed the photo 40% opacity to midgrey.. tricky to do automatically, youd probably have to measure the image c9lour range and grey it by a dynamic ammount

[dobkeratops]

some ideas on validation by switching colours, and summary of how existing annotations fit in

[bbernhard]

Many thanks for the great suggestions - very much appreciated! I really like that approach!

Unfortunately I am quite busy at the moment renovating my new flat, so I probably won't have time in the next few months to tackle such a big feature. At the moment I am mostly doing maintenance (e.g I've updated PostgreSQL to the latest supported major version a few days ago), fixing some small bugs, etc. I hope I find more time soon to work on that :)

[dobkeratops]

good luck with the move and settling in. For the moment the examples I've done will give similar information , but will rely on multi-task training. e.g. there's some images with left/right, and others with arms/legs. This was.a reasonable compromise given what the tool can currently do.

I could ask for some pointers on (a) downloading the existing dataset - to write visualisation and browsing of. it - and uploading annotations (I could try to modify the labelme tool to include this palette.. or use a layer in gimp, then convert into polygon outlines)

this "stable diffusion" AI model that's all the rage at the moment has a well known problem that It creates distorted people. Maybe human annotations like we have here available might help fix it... although it would take many like minded people to have produced 100k's of similar annotations ,we only have a few. thousand here