search

homuler / MediaPipeUnityPlugin

Unity plugin to run MediaPipe
MIT License
1.76k stars 460 forks source link

How to modify NUM_HANDS #24

Closed Densen90 closed 3 years ago

Densen90 commented 3 years ago

Hi there,

in the mediapipe examples i can set the NUM_HANDS variable in the MainActivity.java for hand tracking. Where inside your project can I edit this to have more than two hands detected?

Thanks and cheers!

homuler commented 3 years ago

num_hands is defined in HandTrackingGraph config and you can set value to it through SidePacket (see https://github.com/homuler/MediaPipeUnityPlugin/blob/master/Assets/MediaPipe/Examples/Scripts/HandTracking/HandTrackingGraph.cs#L51).

Densen90 commented 3 years ago

Hi there,

thanks for your help!

I assume I have to change "sidePacket.Emplace("num_hands", new IntPacket(2));" to sidePacket.Emplace("num_hands", new IntPacket(3)); or sidePacket.Emplace("num_hands", new IntPacket(4));

With a value of 1 or 2 it works pretty good, but with a value above that, it either doesn't recognize a third hand or the recognition gets very flicky and jumpy.

Do you have an idea on how to avoid this?

Thank you again!

Densen90 commented 3 years ago

I also updated your repository and tha android hand tracking does work worse now, I don't even get tha position of the finger joints etc. Any idea how I could fix that?

Thanks again and you are doing great work!

homuler commented 3 years ago

With a value of 1 or 2 it works pretty good, but with a value above that, it either doesn't recognize a third hand or the recognition gets very flicky and jumpy.

Can you get more than 2 hand landmark lists? If so, this is only annotation problems. Please change the value of DefaultMaxSize of MultiHandLandmarkListsAnnotation. handlandmarklist-annotation

I don't even get tha position of the finger joints etc.

I confirmed it. I'll investigate it later.

homuler commented 3 years ago

In Progress

I'm investigating it now, but haven't detected the cause yet. I suspect that the input data is broken (e.g. the channel order is wrong, etc...).

homuler commented 3 years ago

memo: At least, PalmDetectionGpu seems to be working, but ThresholdingCalculator returns false (that is, hands are not present in the input) and no landmark data are output.

InferenceCalculator may not be working, or the input tensor (input_tensor) of the calculator may be invalid.

homuler commented 3 years ago

I'm sorry I cannot specify the commit that fixed the problem, but it's fixed at https://github.com/homuler/MediaPipeUnityPlugin/commit/4a656876fe79993630151642665ca6679ea68328 .