Hand gestures for Android phones

3DiVi / nuitrack-sdk

Nuitrack™ is a 3D tracking middleware developed by 3DiVi Inc.

https://nuitrack.com

465 stars 65 forks source link

Hand gestures for Android phones #77

Closed MDVGames closed 2 days ago

MDVGames commented 4 days ago

Hi.

Any plans to support hand gestures for Android phones (like Google MediaPipe)?

I'm interested in a plugin for Unreal Engine 5.

Thanks.

andrey-tsb commented 4 days ago

Hi @MDVGames, Aren't you satisfied with the MediaPipe ? Why not just use it in this case ?

(Nuitrack roadmap isn't set in stone, that's why we always try to work out additional use-cases for Nuitrack through similar questions).

Thanks.

MDVGames commented 4 days ago

Currently, the MediaPipe Plugin for Unreal Engine 5 only works on PC + webcam. It does not work on Android phones.

Would be great if the Nuitrack Plugin for Unreal Engine 5 could export apps to Android phones with hand tracking support.

andrey-tsb commented 3 days ago

Thank you for getting back.

Nuitrack fully supports Android devices with depth sensors attached (like Orbbec Persee / Persee+ or Android phone with an Orbbec/Intel depth sensor), but there is no support for conventional phone camera.
Hand could be tracked as a single joint inside full body skeleton and also simple hand gestures are supported, but there is no support for detailed hand pose ("finger tracking")

We're considering Nuitrack extension in this direction for the long-term, but really don't recommend to rely on this for the nearest projects / plans.

MDVGames commented 3 days ago

Ok. Thank you for the information.