search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.6k stars 5.16k forks source link

Mediapipe iOS hand tracking "flickers" on iPhone X & lower devices #837

Closed Saltb0xApps closed 3 years ago

Saltb0xApps commented 4 years ago

When I run the hand tracker in real time on 1280x720 AVCaptureSession, devices below iPhone X start resetting detection every two or three seconds causing a noticeable flicker.

I've attached a screen recording to show this issue below - https://photos.app.goo.gl/wpv6Vw8eaAF55hQBA

This could be due to the following reasons -

  1. Low RAM scenarios.
  2. Frames being dropped by camera due to intensive processing.

This does not happen in the sample application, but if I do additional processing like applying CoreImage filters on top of the camera, the issue happens almost always.

This issue happens on devices below iPhone X. It works perfectly on iPhone XS, iPhone 11 & 11 Pro, probably because cases 1 & 2 don't happen as often on the more powerful processor.

Is there a way I can run MediaPipe hand tracking on a lower frame rate or low RAM situations?

google-ml-butler[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

google-ml-butler[bot] commented 3 years ago

Closing as stale. Please reopen if you'd like to work on this further.

google-ml-butler[bot] commented 3 years ago

Are you satisfied with the resolution of your issue? Yes No