Hand Gesture Recognizer unable to use GPU inference

[JehanJaye]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Nvidia Tegra Orin aarch64

MediaPipe Tasks SDK version

0.10.2

Task name (e.g. Image classification, Gesture recognition etc.)

Gesture Recognition

Programming Language and version (e.g. C++, Python, Java)

Python

Describe the actual behavior

I have been using Mediapipe's Gesture Recognition library in Python, and the CPU inferencing works all right. However, I noticed that version (0.10.2) of the Gesture Recognition module does not support GPU acceleration, which could significantly enhance its performance, especially on devices with GPUs.

Describe the expected behaviour

Integrate GPU support into the Gesture Recognition module, enabling users to benefit from enhanced performance. Or if it's already available, please point out where to configure that in the library since changing the delegate to GPU only does not help utilizing the GPU.

self.options = GestureRecognizerOptions(
            base_options=BaseOptions(model_asset_path=self.model_path, delegate="GPU"),
            running_mode=VisionRunningMode.VIDEO,
        )

Standalone code/steps you may have used to try to get what you need

1. Built a GPU-compatible whl file for aarch64 architecture.
2. Tested the whl file on my system and observed performance improvements in other modules.
3. Verified that the Gesture Recognition module does not currently support GPU acceleration.

Other info / Complete Logs

I20230821 16:28:38.537245 25612 gl_context.cc:342] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 35.3.1), renderer: NVIDIA Tegra Orin (nvgpu)/integrated
Camera rgb: 7096 Frames [11:50, 10.66 Frames/sW20230821 16:28:38.773761    28 gesture_recognizer_graph.cc:128] Hand Gesture Recognizer contains CPU only ops. Sets HandGestureRecognizerGraph acceleration to Xnnpack.
I20230821 16:28:38.776032    28 hand_gesture_recognizer_graph.cc:249] Custom gesture classifier is not defined.
I20230821 16:28:38.783733    28 gl_context_egl.cc:84] Successfully initialized EGL. Major : 1 Minor: 5

[kuaashish]

@JehanJaye,

Currently, We do not offer any official support, libraries, modules to build into Jetson. There are plugin from the community is quite up-to-date which might help in this context But For now, We do not have a plan to support Jetson and does not provide support in how to use them.

Just a note, We have recently rolled out the support of Raspberry OS 64-bit in v0.10.3 as noted and example can be seen here.

Thank you

[JehanJaye]

@kuaashish

Thank you for your response. I'd like to add a bit more detail to my issue regarding my experience with mediapipe-gpu on Jetson.

I've actually managed to successfully build and utilize the mediapipe-gpu ( v0.10.2 ) library on my Jetson. While I understand that the official Gesture Recognition vision module doesn't currently support GPU acceleration as of version 0.10.2, my intention is to see if there's any potential plan or possibility to extend GPU support to the Gesture Recognition module in future releases.

Any further guidance or insights you can provide on this matter would be appreciated.

[kuaashish]

@JehanJaye,

Please go through the comment(4707) for better understanding. Thank you

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[JehanJaye]

So the mediapipe GPU building was successful and works. However, since the gesture recognition binary graph for GPU is not currently available, Hand Gesture Recognizer only runs with the CPU. Also, note that the other modules run seamlessly with the GPU in my platform.

@kuaashish When do you plan to release the GPU version for Hand Gesture recognition?

Thank you

[JehanJaye]

@schmidt-sebastian When do you plan to release the gesture_recognizer_graph with GPU ops?

[il-nietos]

@JehanJaye I am currently trying to build MediaPipe for Jetson Orin, would love any hints on building it for Jetson platforms. Would you mind specifying which changes you made to the mediapipe source code and how you installed opencv to get a working version?

[Trenton-Ruf]

I am also seeing that the hand gesture solution cannot be ran GPU yet. Is there a plan to make a gesture_recognizer_graph with GPU ops?

Hand Gesture Recognizer contains CPU only ops. Sets HandGestureRecognizerGraph acceleration to Xnnpack.

When setting delegate=BaseOptions.Delegate.GPU it seems to try make a GPU delegate, but then reverts to CPU since the ops do not exist.

I0000 00:00:1720855483.067228 2544335 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1720855483.156018 2544497 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 555.58.02), renderer: NVIDIA GeForce RTX 4090/PCIe/SSE2
W0000 00:00:1720855483.156332 2544335 gesture_recognizer_graph.cc:129] Hand Gesture Recognizer contains CPU only ops. Sets HandGestureRecognizerGraph acceleration to Xnnpack.
I0000 00:00:1720855483.156959 2544335 hand_gesture_recognizer_graph.cc:250] Custom gesture classifier is not defined.
INFO: Created TensorFlow Lite delegate for GPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
W0000 00:00:1720855483.212475 2544520 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.
W0000 00:00:1720855483.212555 2544520 inference_feedback_manager.cc:114] Feedback manager requires a model with a single signature inference. Disabling support for feedback tensors.

[Trenton-Ruf]

Well I feel silly now.

After reading through this issue I see that both delegates being created is expected when specifying delegate=BaseOptions.Delegate.GPU https://github.com/google-ai-edge/mediapipe/issues/4711

At first I didn't see any performance differences between them, but it turned out to be an issue with my camera having a low frame-rate...

For testing I sent both delegates a static image and now see a bout a 5.5x performance increase on the GPU Delegate.

Also confirmed it was using the GPU with nvidia-smi.

[kuaashish]

Hi @JehanJaye,

Our Gesture Recognizer Task API supports GPU in Python for macOS and regular Ubuntu starting from version https://github.com/google-ai-edge/mediapipe/releases/tag/v0.10.8. Could you please adjust as per the comment https://github.com/google-ai-edge/mediapipe/issues/4712#issuecomment-2227178861 and update us on the status?

Thank you!!

[JehanJaye]

Hi @kuaashish,

The issue is fixed now. Thanks!

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No