Closed davimedio01 closed 2 years ago
Sorry, the document is stale. NativePacketCallback
needs to return StatusArgs
now.
https://github.com/homuler/MediaPipeUnityPlugin/blob/adb2d908cb6e7b82950c8bfa51c493760a6293b8/Packages/com.github.homuler.mediapipe/Runtime/Scripts/Framework/CalculatorGraph.cs#L16
However, if you use OutputStream
, you don't need to implement a NativePacketCallback
.
See OutputStream#AddListner
for more details.
https://github.com/homuler/MediaPipeUnityPlugin/blob/adb2d908cb6e7b82950c8bfa51c493760a6293b8/Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/OutputStream.cs#L141-L148
Here's a Hand Tracking sample code. https://github.com/homuler/MediaPipeUnityPlugin/blob/adb2d908cb6e7b82950c8bfa51c493760a6293b8/Assets/MediaPipeUnity/Samples/Scenes/Hand%20Tracking/HandTrackingGraph.cs#L53-L57 https://github.com/homuler/MediaPipeUnityPlugin/blob/adb2d908cb6e7b82950c8bfa51c493760a6293b8/Assets/MediaPipeUnity/Samples/Scenes/Hand%20Tracking/HandTrackingSolution.cs#L50 https://github.com/homuler/MediaPipeUnityPlugin/blob/adb2d908cb6e7b82950c8bfa51c493760a6293b8/Assets/MediaPipeUnity/Samples/Scenes/Hand%20Tracking/HandTrackingSolution.cs#L103-L106
Thank you for the information!
I'm trying to implement it, but I'm still a little lost on how to implement this event in the graph coming from the "CalculatorGraph"
class (as in my code, since it doesn't have this event). Performing the "AddListener"
of my "OutputStream"
is also not resulting in anything...
I wonder if I would necessarily have to implement three classes like in the example scene?
I'm still a little lost on how to implement this event in the graph coming from the "CalculatorGraph" class
Which event?
What you need to implement is a callback function (see HandTrackingSolution#OnHandLandmarksOutput
) and you don't need to implement any events (you don't need to implement HandTrackingGraph#OnHandLandmarksOutput
).
I wonder if I would necessarily have to implement three classes like in the example scene?
I think you can implement it in a single class.
Thank you very much! It's working perfectly now!
For those who have doubts, just use the "AddListener"
function of "OutputStream"
and put the callback in it, as below.
_handLandmarksStream.AddListener(OnHandLandmarksOutput);
and
private void OnHandLandmarksOutput(object stream, OutputEventArgs<List<NormalizedLandmarkList>> eventArgs)
{
_handLandmarksAnnotationController.DrawLater(eventArgs.value);
}
Hey, I'm also trying to implement the same but using hair segmentation. I have the folllowing
_graph = new CalculatorGraph(_configAsset.text);
var outputHairMask = new OutputStream<ImageFramePacket, ImageFrame>(_graph, "hair_mask");
outputHairMask.AddListener(OnHandLandmarksOutput);
_graph.StartRun().AssertOk();
stopwatch.Start();
while (true)
{
_inputTexture.SetPixels32(_webCamTexture.GetPixels32(_inputPixelData));
var imageFrame = new ImageFrame(ImageFormat.Types.Format.Srgba, _width, _height, _width * 4, _inputTexture.GetRawTextureData<byte>());
var currentTimestamp = stopwatch.ElapsedTicks / (System.TimeSpan.TicksPerMillisecond / 1000);
_graph.AddPacketToInputStream("input_video", new ImageFramePacket(imageFrame, new Timestamp(currentTimestamp))).AssertOk();
yield return new WaitForEndOfFrame();
}
And it runs for a couple seconds but then I get this error
MediaPipeException: NOT_FOUND: Graph has errors:
OutputStream with id 1 is not found
Mediapipe.Status.AssertOk () (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Framework/Port/Status.cs:149)
Mediapipe.Unity.Tutorial.HairSolution+<Start>d__14.MoveNext () (at Assets/Project/Materials/HairSolution.cs:82)
UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) (at <f1212ad1dec44ce7b7147976b91869c3>:0)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr, Boolean&)
var outputHairMask = new OutputStream<ImageFramePacket, ImageFrame>(_graph, "hair_mask");
This error occurs because the OutputStream
object has been already GCed.
Please store OutputStream
objects to member variables.
private OutputStream<ImageFramePacket, ImageFrame> outputHairMask;
it worked. Thanks a lot
Plugin Version or Commit ID
v.0.10.1
Unity Version
2021.3.3f1
Your Host OS
Manjaro KDE with Kernel 5.4.207-1
Target Platform
UnityEditor, Android
Description
Hey guys! First, thanks for all the support for this amazing plugin. I'm from Brazil, so my comments in the attached code are in Brazilian Portuguese, since they are test scripts before the final project for my college. So ignore them for now.
Let's go to the issue! I was trying to build a solution for Android (OpenGLES) involving ARFoundation with Hand Tracking (specifically, the output obtained by MULTI_HAND_LANDMARKS) offered by the plugin and, as stated in #343, it is necessary to use the Asynchronous API described in Advanced Topics. However, as shown in the images below, the
ObserveOutputStream
function is not working when I reference myHandLandmarkCallback
function as a parameter, even following the steps described in the wiki and checking the files of the "Hand Tracking" example scene.And the MediaPipe OpenGLES Graph (output "4" for MULTI_HAND_LANDMARKS):
Code to Reproduce the issue
Additional Context
No response