Ubuntu multi_hand_tracking_desktop GPU and CPU

[pranayagra]

I am trying to run multi_hand_tracking_desktop CPU/GPU (I have GPU but I receive the same results in both). The bazel build command works for both CPU/GPU, as I receive the expected print:

Target //mediapipe/examples/desktop/multi_hand_tracking:multi_hand_tracking_cpu up-to-date:

bazel-bin/mediapipe/examples/desktop/multi_hand_tracking/multi_hand_tracking_cpu

OR

Target //mediapipe/examples/desktop/multi_hand_tracking:multi_hand_tracking_gpu up-to-date:

bazel-bin/mediapipe/examples/desktop/multi_hand_tracking/multi_hand_tracking_gpu

However, the second portion of the command does not work on either:

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/multi_hand_tracking/multi_hand_tracking_gpu \ --calculator_graph_config_file=mediapipe/graphs/hand_tracking/multi_hand_tracking_mobile.pbtxt

It outputs: (FILE INFO) I1125 22:11:02.310927 3671 demo_run_graph_main_gpu.cc:56] Initialize the calculator graph. I1125 22:11:02.313064 3671 demo_run_graph_main_gpu.cc:60] Initialize the GPU. I1125 22:11:02.325788 3671 gl_context_egl.cc:158] Successfully initialized EGL. Major : 1 Minor: 4 I1125 22:11:02.328773 3685 gl_context.cc:233] GL version: 3.2 (OpenGL ES 3.2 Mesa 19.2.1) I1125 22:11:02.328826 3671 demo_run_graph_main_gpu.cc:66] Initialize the camera or load the video. I1125 22:11:02.545353 3671 demo_run_graph_main_gpu.cc:91] Start running the calculator graph. I1125 22:11:02.546195 3671 demo_run_graph_main_gpu.cc:96] Start grabbing and processing frames. INFO: Created TensorFlow Lite delegate for GPU.

And then opens another screen with mediapipe, but instead of my webcam, the screen is white with the light on my computer on, indicating that the webcam is on. I know the webcam itself works because running it separately works. This happens on both CPU and GPU.

[eknight7]

Hi, thanks for reporting this, we are looking into a similar issue: https://github.com/google/mediapipe/issues/269.

[alizahidraja]

Try checking the webcam permission

[pranayagra]

Try checking the webcam permission

How should I go about this?

[alizahidraja]

Try checking the webcam permission

How should I go about this?

You can try using sudo with the command or search for permission of webcam on google, i searched myself and found multiple answers, havent tried any so don't want to mislead you. the most convincing was "sudo chmod 777 /dev/video*" Do give it a try

[pranayagra]

Try checking the webcam permission

How should I go about this?

You can try using sudo with the command or search for permission of webcam on google, i searched myself and found multiple answers, havent tried any so don't want to mislead you. the most convincing was "sudo chmod 777 /dev/video*" Do give it a try

Does not seem to solve the issue

[pranayagra]

Similar to #269, object detection works fine, which means it can pull up the webcam correctly here.

[eknight7]

Hi @pranayagra thanks for confirming that.

Can you also let me know if hair segmentation is not working? What about face detection?

Also, we have graphs for hand detection only: https://github.com/google/mediapipe/blob/master/mediapipe/graphs/hand_tracking/hand_detection_desktop_live.pbtxt can you try to add respective BUILD targets and see if hand_detection_desktop_live works for you? Thanks!

[pranayagra]

Hair segmentation (GPU): white screen (similar issue) Face detection (CPU): works - see red box with camera and some location points Face detection (GPU): works - see red box with camera and some location points

Not sure what you mean by adding respective BUILD targets to see if that file works, but Hand detection (CPU): white screen (similar issue) Hand detection (GPU): white screen (similar issue)

[wolterlw]

similar issue. Build process completes without any errors, on launch gets stuck right after INFO: Created TensorFlow Lite delegate for GPU. I've also tried to use a .mp4 file as input. Gets stuck in the same place anyway. Also it doesn't create a window, but the webcam light turns on.

[eknight7]

Can you do a fresh pull and try? We pushed a few changes today.

[wolterlw]

@eknight7 works on my machine. However it has strange glitches. Here's an example

[wolterlw]

@eknight7 also could you or anybody on the team elaborate a little on how to build a tflite GPU delegate for desktop? I'm developing a Python wrapper for hand tracking models to prototype an improved tracking pipeline. But as of now I can only use a CPU delegate, and it's quite slow to run experiments with. So I'd really appreciate a little help.

[lorenzob]

Same here, face detection works (with CPU and GPU), hand and multi hands do not (with CPU and GPU). A mediapipe window appears in the taskbar, webcam light turns on, no cpu usage.

With latest commit (137867d) single hand tracking works while multi hand tracking shows the ghost window as before.

[pranayagra]

The changes seem to fix the issue. Thanks very much!

[MitchDtown]

@eknight7 works on my machine. However it has strange glitches. Here's an example

Same problem here

[eknight7]

This particular glitch is reproducible for other examples, such as single hand detection, and is related to rendering in OpenCV. We will look into it, thank you for reporting.

[wolterlw]

@eknight7 ok, but what about the tflite GPU delegate?

[mcclanahoochie]

The hanging issue sounds related to a bug that was fixed recently in the latest release, can you try updating?

[jiuqiant]

We are closing this issue for now due to lack of activity.