Full body detection with max accuracy on desktop and GPU

[rdisabato]

Hi,

What I am trying to achieve is the execution of the Holistic solution at maximum performance. Next I would like to add iris detection as well (if it is not already included in the Holistic solution by setting the refine_face_landmarks = true parameter). To do this I am using a desktop with ubuntu 20.04 and an Nvidia video card which supports OpenGL ES 3.2.

For maximum performance I seem to have realized that I cannot use Python, because Python interfaces do not support GPU. For this I am only using C++.

Currently I have been able to run Holistic with GPU on C++, but I notice that the model that is used is not "heavy" (the detections are not very accurate). So my first question is: How do I run the Holistic solution via C++ by setting the Holisitc configuration parameters (such as modelComplexity and refineFaceLandmarks)?

I also ask if you think my approach is right for my goals, or if I should use other approaches.

System information (Please provide as much relevant information as possible)

• Have I written custom code (as opposed to using a stock example script provided in Mediapipe): no
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04, Android 11, iOS 14.4): Ubuntu 20.04
• MediaPipe version: 0.8.9
• Bazel version: 4.2.1
• Solution (e.g. FaceMesh, Pose, Holistic): Holistic, Iris
• Programming Language and version ( e.g. C++, Python, Java): C++

[rdisabato]

Thanks a lot @sgowroji ! I will try by settings those values. But I have doubts about what to do exactly.

Do I have to directly set the configuration parameters in the pbtxt files you indicated? For example, in the holistic_landmark_gpu.pbtxt file, to set the complexity, I have to replace the line: input_side_packet: "MODEL_COMPLEXITY: model_complexity" with the line: input_side_packet: "MODEL_COMPLEXITY: 2" ? Or maybe insert a new line like: model_complexity = 2 ?

Or when i run mediapipe do i have to use a command line parameter? For example --model_complexity = 2?

Or maybe the command line parameter must be something like: --input_side_packets = model_complexity = 2, refine_face_landmark = true ... ?

Finally, should I specify the holistic_landmark_gpu.pbtxt file in the modules folder from the command line (with the --calculator_graph_config_file parameter)? Or should I use the holistic_tracking_gpu.pbtxt file in the graphs folder? I didn't understand the difference.

Sorry for all these questions, but I can't find accurate information in the documentation.

Thanks so much again!

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[rdisabato]

Please do not close this support request.

I'm waiting for a reply. In the meantime I tried to figure out how to do it, but I didn't succeed and I didn't find the guides of Mediapipe or Bazel useful.

In summary the question is: how do I specify the configuration parameters of the "face_landmark" (ie "with_attention") and "pose_landmark" (ie "model_complexity") solutions when I run the "holistic" solution?

I currently use the start command: bazel-bin/mediapipe/examples/desktop/holistic_tracking/holistic_tracking_gpu --calculator_graph_config_file=mediapipe/graphs/holistic_tracking/holistic_tracking_gpu.pbtxt

How do I declare the necessary side packets from the command line?

[ayushgdev]

Hello @rdisabato We are revisiting older issues. To run the holistic tracking with the config parameters requested, you need to make the following changes:

1. mediapipe/examples/desktop/holistic_tracking/BUILD uses demo_run_graph_main_gpu.cc as deps for holistic_tracking_gpu. So modify this file to supply input side packets to the holistic graph. Check [simple_run_graph_main.cc] (https://github.com/google/mediapipe/blob/master/mediapipe/examples/desktop/simple_run_graph_main.cc#L111) for reference.

2. holistic_tracking_gpu.pbtxt uses HolisticLandmarkGpu which is a subgraph. This subgraph internally calls PoseLandmarkGpu which uses the model_complexity and FaceLandmarksFromPoseGpu which uses refine_face_landmarks parameter to determine the model complexity and attention network usage. So in order to provide these arguments, add the following lines to the HolisticLandmarkGpu node in holistic_tracking_gpu.pbtxt file.

   node {
   calculator: "HolisticLandmarkGpu"
   ...
   input_side_packet: "MODEL_COMPLEXITY:model_complexity"
   input_side_packet: "REFINE_FACE_LANDMARKS:refine_face_landmarks"
   ...

3. Add following to the start of holistic_tracking_gpu.pbtxt since this is the entry point of the graph and it should know it has some input side packets to expect:

   input_side_packet: "model_complexity"
   input_side_packet: "refine_face_landmarks"

4. To run the graph with arguments, use the --input_side_packets="model_complexity=0/1/2, refine_face_landmarks=True/False" as the command line arguments in the run command bazel-bin/mediapipe/examples/desktop/holistic_tracking/holistic_tracking_gpu --calculator_graph_config_file=mediapipe/graphs/holistic_tracking/holistic_tracking_gpu.pbtxt

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No