homuler / MediaPipeUnityPlugin

Unity plugin to run MediaPipe
MIT License
1.8k stars 467 forks source link

Basic subscriber / event listener example #114

Open Wayne-nulitics opened 3 years ago

Wayne-nulitics commented 3 years ago

@homuler Thank you for explaining the purpose of this port and use of mediapipe::CalculatorGraph, that we are looking for 'n a sample code snippet in C# that subscribes to the detection/graph calculation, we've been able to "track it down" to:

public class HandTrackingGraph : DemoGraph { private OutputStreamPoller handLandmarksStreamPoller; ... var handLandmarks = isHandLandmarksPresent ? FetchNextHandLandmarks() : new List(); ... private void RenderAnnotation(WebCamScreenController screenController, HandTrackingValue value) { // NOTE: input image is flipped GetComponent().Draw( screenController.transform, value.HandLandmarkLists, value.Handednesses, value.PalmDetections, value.PalmRects, true); }

But how, from another scripts, can we subscribe the receive the value.HandLandmarkLists and value.Handednesses as they are being updated.

ie. we can't figure out how to subscribe to the var handLandmarks or something that provides that with handedness.

I've mapped out the basic flow through the demo as follow, just the key points, assume we can add something similar to the RenderAnnotation function? image

We are well versed in using it in Python, and the hand_landmark detection from here https://google.github.io/mediapipe/solutions/hands.html

Any help would be appreciated.

homuler commented 3 years ago

296 adds a working example.

https://github.com/homuler/MediaPipeUnityPlugin/blob/7bb877de4887ad4bf23c72b39649c41bb3650d54/Assets/Mediapipe/Samples/Scenes/Hand%20Tracking/HandTrackingSolution.cs#L79

I will add documentation on this later.