google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://mediapipe.dev
Apache License 2.0
26.13k stars 5.04k forks source link

Face Landmark Detection grid is shifted #5431

Open Merligus opened 1 month ago

Merligus commented 1 month ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Android 11

Mobile device if the issue happens on mobile device

Samsung Galaxy A02

Browser and version if the issue happens on browser

Google Chrome 125.0.6422.53

Programming Language and version

Javascript

MediaPipe version

No response

Bazel version

No response

Solution

Face Landmark Detection

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

When I choose the grid renderer in the face landmarker demo, the grid is shifted to the top left by a few pixels

Describe the expected behaviour

The grid should be placed exactly on the face.

Standalone code/steps you may have used to try to get what you need

https://mediapipe-studio.webapps.google.com/studio/demo/face_landmarker

Other info / Complete Logs

W0522 18:28:37.938080 1885424 landmarkprojection_calculator.cc:189] Using NORM_RECT without IMAGE_DIMENSIONS is only supported for the square ROI. Provide IMAGE_DIMENSIONS or use PROJECTION MATRIX. 
onFilePicked: {"0":{}} 
Graph finished closing successfully. 
I0522 18:29:37.060000 1885424 gl_context webgl.cc:132] Successfully destroyed WebGL context with handle 2
W0522 18:29:37.141080 1885424 face_landnarker_graph.cc.180] Sets FaceBlendshapesGraph acceleration to xnnpack by default. T0522 18:29:37.502000 1885424 gl_context.cc:374] GL version: 3.0 (OpenGL ES 3.0 (WebGL 2.0 (OpenGL ES 3.0 Chromium))), renderer: WebKit WebGL 
Graph successfully started running
INFO: Created TensorFlow Lite XNNPACK delegate for CPU. 
INFO: XNNPack weight cache not enabled.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU
INFO: XNNPack weight cache not enabled.
Merligus commented 1 month ago

a5286a53-97e5-4fbd-8f7f-517b17a34a52

kuaashish commented 1 month ago

Hi @Merligus,

We need your help to test this on a another device, if possible. We believe the issue might be specific to the device you are currently using. Please let us know your findings so we can further investigate this issue.

Thank you!!

Merligus commented 1 month ago

Ok. I'll check if the same error happens on another device. I'll let you know as soon as I discover.

Merligus commented 1 month ago

Hey, I found this one. Moto E22 Android 12 Chrome 125.0.6422.52.

kuaashish commented 4 weeks ago

Hi @mbrenon,

I successfully ran the Face Landmarker Mediapipe Grid Render demo on a Pixel 6a without any grid shifting issues. It appears this problem is device-specific. Attached is a screenshot for your reference. Could you please investigate why the rendering differs across devices?

Screenshot_20240607_124132

Thank you!!