search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.57k stars 5.16k forks source link

Hand Gestures types #2386

Closed Nickil21 closed 3 years ago

Nickil21 commented 3 years ago

Is it possible to identify hand gesture types such as axis (vertical, horizontal), shape (arced/straight), direction (upward/downward/leftward/rightward), and handedness (left/right/both hands) using mediapipe for a video frame as the input? I know currently that it supports the handedness gesture type, but I wondered if something exists for the other types.

AmitMY commented 3 years ago

You can use some clever math to figure out the axis (plane), direction (view/facing), and rotation https://www.youtube.com/watch?v=L95mA_h0CjA

And maybe use a simple hand shape classification model https://www.youtube.com/watch?v=pCKRWSNIaNQ Although it is not very accurate at this point.

Math + model inference code: https://github.com/sign-language-processing/playground/blob/master/src/app/modules/sign-writing/hands.service.ts

sgowroji commented 3 years ago

Hi @Nickil21, Please refer this blog for Gesture Recognition https://ai.googleblog.com/2019/08/on-device-real-time-hand-tracking-with.html

HungDuong1998 commented 3 years ago

Hi @sgowroji , It said in the blog that the Gesture Recognition is already published

The existing pipeline supports counting gestures from multiple cultures, e.g. American, European, and Chinese, and various hand signs including “Thumb up”, closed fist, “OK”, “Rock”, and “Spiderman”.

But I can't seem to find it, can you help?

Many thanks