search

snavas / GECCO

Gesture-Enabled Remote Communication and Collaboration
MIT License
5 stars 0 forks source link

utilizing edges and contours #14

Open PaulaScharf opened 3 years ago

PaulaScharf commented 3 years ago

opencv provides a function called Canny to detect edges. maybe this could be used to improve the contours of the hand

PaulaScharf commented 3 years ago

This is the result after calculating the edges and then getting the contours: contours

I think it has some potential I just need to figure out a way to use it. Biggest problem is that the contours are not perfect (eg often they do not close completly).

PaulaScharf commented 3 years ago

There is a machine learning approach called graph-cut that does semantic segmentation using edge detection but I am not sure if it is fast enough.

PaulaScharf commented 3 years ago

Interesting aproach to filling the mask until edges here. It will probably be too slow though...

PaulaScharf commented 3 years ago

I added a mode just now where only the edges of a hand are visualized: edges

The procedure is: first calculate edges for the entire frame, then create the hand mask as usual but heavily dilate it and lastly use the mask to cut out the edges.

I think the contours of these edges are often better than the mask. The outline is more realistic. However quality of the result is still very dependent on the mask. For example if the mask is missing a finger this will also be cut out of the edges.

A big draw back is the low visibility of the edges. Maybe I can figure out an intelligent way to set the color of the edges.

PaulaScharf commented 3 years ago

I tried filling the edges with the watershed function. The results are fine, but not better than what we currently have. watershed

Edit: I improved the watershed again today. I would say that the results look promising. I especially like how the fingers are segmented. The framerate is awful but the code is also really messy at the moment so there is probably room for improvements. watershed2