search

virtual-puppet-project / vpuppr

VTuber application made with Godot 4
Mozilla Public License 2.0
721 stars 60 forks source link

[FEATURE]: Mediapipe support #185

Open you-win opened 1 year ago

you-win commented 1 year ago

Description

Add mediapipe as a supported tracker. Mediapipe holistic + iris tracking provides full face and body tracking.

Why is this needed

Mediapipe only requires a webcam to track a person's face and body.

Additional information

GDMP - A gdnative integration for mediapipe that works on both windows and linux

fonck838 commented 1 year ago

godot_mediapipe_module - would also work to add Mediapipe into the project

CartoonFan commented 10 months ago

Has there been any progress on this? It seems like it'd be a pretty significant upgrade :thinking:

fonck838 commented 10 months ago

I have been able to build the module into the project, and i made a basic extension GUI that interrogates just like all the other extensions but i get lost when trying to connect the tracking data from media pipe into the rest of the program. I also never figured out the best way to translate the media pipe tracking data, there are a lot of examples online on how the math works but that would also need to be done

you-win commented 10 months ago

The latest commit on master adds preliminary mediapipe support (that's still broken on vpuppr's side) via GDMP.

The relevant files/lines are: