creativeIKEP / BlazePoseBarracuda

BlazePoseBarracuda is a human 2D/3D pose estimation neural network that runs the Mediapipe Pose (BlazePose) pipeline on the Unity Barracuda with GPU.
Apache License 2.0
198 stars 31 forks source link

Detecter returns NaN for x,y,z in each (x, y, z, w) Pose World Landmark #15

Closed deeptronos closed 1 year ago

deeptronos commented 2 years ago

Hi there!

BlazePose runs just fine in the Unity Project from this repository on my computer. I'm trying to integrate BlazePoseBarracuda into a different Unity Project, but I'm experiencing odd behavior from the PoseVisualizer/PoseVisualizer3D scripts in that project.

I've exported all of the assets from this original Unity Project as a .unitypackage file. After that, I install the packages required for BlazePoseBarracuda using this technique in the README in the new Project, and there seems to be no problems with that process. Then, I import my BlazePoseBarracuda .unitypackage in the new Project. When I run a demo scene in the new project, e.g. the 3DSampleScene, no pose skeleton appears & the Debug.Log() calls in PoseVisuallizer3d.cs output Vector4s of the format (NaN, NaN, NaN, w), where w is the landmark position visibility score. w seems to consistently be a number, but the first three coordinates (the x, y, z of each pose landmark) are NEVER a number. They're only NaN. The 33rd index data is always consistently (0.0, 0.0, 0.0, 0.0) too.

See the below screenshot for an example (I was visible to the webcam for this capture, so, if BlazePose were working, it would have been able to detect a pose.) Screenshot

Technical details: I'm running MacOS 12.3/Unity 2020.3.33f1. I'm using Barracuda 2.1.0-preview (because it's required for another NN pose detection implementation. I've tried other ver of Barracuda with BlazePoseBarracuda in this Unity Project and had same result.) My build target is Android. My computer has an Intel CPU and an AMD GPU. Graphics APIs: OpenGLES2 & 3 are enabled, but Metal I think since I'm on a Mac?

creativeIKEP commented 2 years ago

@deeptronos The Nan phenomenon could not be reproduced in my environment (MacOS 12.5, M1 Max chip MacBook & Barracuda updated to 2.1.0-preview)

deeptronos commented 2 years ago

@creativeIKEP I believe this error may have something to do with Metal on Intel or AMD hardware... but I also suspect that my package installs are not configured entirely correctly.

creativeIKEP commented 1 year ago

@deeptronos Can you try the following and see if the issue still occurs?

Supplement: You have a project for OpenGL ES on Android as your build target, but Unity Barracuda's GPU inference does not support OpenGL ES on Android.

creativeIKEP commented 1 year ago

I believe this issue is resolved. I close this issue.