Python API with GPU on Jetson Xavier NX

[kelvinxuande]

I have diligently followed the instructions on a Jetson Xavier NX found here and managed to successfully use the python API (specifically .solutions) for pose estimation to query the keypoints in an image. I would like to ask - in such a scenario, is the GPU enabled and utilised? i.e. are there seperate bazel build procedure/ configurations to enable and utilise the GPU?

Thanks and best regards.

[jiuqiant]

The prebuilt MediaPipe Python wheels only have the cpu graphs and running those solutions doesn't require GPU. It's possible to modify the config and build a local gpu version of the Python package. See https://github.com/google/mediapipe/issues/1042. Note that that post assumes the version is v0.7.9 not the current version.

[kelvinxuande]

@jiuqiant thank you so much for your reply -

I have given the instructions found in #1042 a shot but met with the following error. Note that mediapipe was built and installed successfully into a virtual environment.

Code:

import mediapipe as mp
import cv2
mp_pose = mp.solutions.pose
pose = mp_pose.Pose(static_image_mode=False, min_detection_confidence=0.5, min_tracking_confidence=0.5)

cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

while cap.isOpened():
  ret, image = cap.read()
  if not ret:
    print("Ignoring empty camera frame.")
    # If loading a video, use 'break' instead of 'continue'.
    continue

  image.flags.writeable = False
  results = pose.process(image)
  image_height, image_width, _ = image.shape

  if not results.pose_landmarks:
    continue
  nose_tuple = (int(results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].x * image_width), int(results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].y * image_height))
  # Radius of circle
  radius = 5
  # Blue color in BGR
  color = (255, 0, 0)
  # Line thickness of 2 px
  thickness = 2
  image = cv2.circle(image, nose_tuple, radius, color, thickness)

  cv2.imshow('MediaPipe Pose', image)
  k = cv2.waitKey(1) & 0xFF
  # press 'q' to exit
  if k == ord('q'):
      break
pose.close()
cap.release()

Error thrown:

Traceback (most recent call last):
  File "mediapipe-test.py", line 1, in <module>
    import mediapipe as mp
  File "/home/xavier/.virtualenvs/mediapipe_gpu/lib/python3.6/site-packages/mediapipe/__init__.py", line 16, in <module>
    from mediapipe.python import *
  File "/home/xavier/.virtualenvs/mediapipe_gpu/lib/python3.6/site-packages/mediapipe/python/__init__.py", line 18, in <module>
    import mediapipe.python.solutions.face_mesh
  File "/home/xavier/.virtualenvs/mediapipe_gpu/lib/python3.6/site-packages/mediapipe/python/solutions/__init__.py", line 18, in <module>
    import mediapipe.python.solutions.face_mesh
  File "/home/xavier/.virtualenvs/mediapipe_gpu/lib/python3.6/site-packages/mediapipe/python/solutions/face_mesh.py", line 38, in <module>
    from mediapipe.python.solution_base import SolutionBase
  File "/home/xavier/.virtualenvs/mediapipe_gpu/lib/python3.6/site-packages/mediapipe/python/solution_base.py", line 49, in <module>
    from mediapipe.python._framework_bindings import calculator_graph
ImportError: /home/xavier/.virtualenvs/mediapipe_gpu/lib/python3.6/site-packages/mediapipe/python/_framework_bindings.cpython-36m-aarch64-linux-gnu.so: undefined symbol: _ZN2cv3MatC1Ev

[jiuqiant]

"undefined symbol: _ZN2cv3MatC1Ev" means opencv is not linked correctly. Do you install opencv locally and build mediapipe python by python3 setup.py gen_protos && python3 setup.py install --link-opencv. If you plan to compile opencv with mediapipe, try python3 setup.py gen_protos && python3 setup.py bdist_wheel.

[jiuqiant]

We are closing this issue for now due to lack of activity.

[only-tao]

I have got the cpu version mediapipe , but now I want to get the gpu version then I follow the #1042 ,

add the --copt=-DMESA_EGL_NO_X11_HEADERS --copt=-DEGL_NO_X11 in two places, but I get a error: when I run the code: python3 setup.py gen_protos && python3 setup.py bdist_wheel

the error like this : `nx@nx-desktop: -/mediapipe ／usr/bin/gcc-U_FORTIFY_SOURCE-fstack-protector-Wall-Wunused-but-set-parameter -Wno-free-nonh －object-fno-omit-frame-pointer -gθ -02 '-D_FORTIFY_SOURCE=1' -DNDEBUG-ffunction-sections. . . (r ining 69 argument (s) skipped)

Use--sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command r/bin/gcc -U FORTIFY SOURCE -fstack-protector -Wall -Wunused-but-set-parameter-Wno-free-nonheap ect-fno-omit-frame-pointer-g0-02 '-D_FORTIFY_SOURCE=1'-DNDEBUG -ffunction-sections. . . (remai g 69 argument (s) skipped)

Use--sandbox_debug to see verbose messages fron the sandbox mediapipe/calculators/tensor/image_to_tensor_converter_gl_buffer.cc: In function 'absl: :lts_2020 23: : StatusOr <std: :unique_ptr > mediapipe: : CreateImageToGLBuff nsorConverter (mediapipe: : CalculatorContext*,bool,mediapipe: : BorderMode) ': mediapipe/calculators/tensor/image_to_tensor_converter_gl_buffer.ck: 342: 10:error:could not con ＇result' from 'std: :unique_ptr <nediapipe: : {anonymous}::GlProcessor,std::default_delete<mediapi ｛anonymous} : : GLProcessor> > 「to 'absl: :lts_2020_09_23: : Status0r <std: :unique_ptr <mediapipe: : Image nsorConverter> > return result; Target//mediapipe/modules/face_detection:face_detection_short_range_cpu failed to build ICes to see the command Lines of failed build steps. ` How can I do next？ in jeton xavier ubuntu 18.04

[khanhnd0408]

I have got the cpu version mediapipe , but now I want to get the gpu version then I follow the #1042 ,

add the --copt=-DMESA_EGL_NO_X11_HEADERS --copt=-DEGL_NO_X11 in two places, but I get a error: when I run the code: python3 setup.py gen_protos && python3 setup.py bdist_wheel

the error like this : `nx@nx-desktop: -/mediapipe ／usr/bin/gcc-U_FORTIFY_SOURCE-fstack-protector-Wall-Wunused-but-set-parameter -Wno-free-nonh －object-fno-omit-frame-pointer -gθ -02 '-D_FORTIFY_SOURCE=1' -DNDEBUG-ffunction-sections. . . (r ining 69 argument (s) skipped)

Use--sandbox_debug to see verbose messages from the sandbox gcc failed: error executing command r/bin/gcc -U FORTIFY SOURCE -fstack-protector -Wall -Wunused-but-set-parameter-Wno-free-nonheap ect-fno-omit-frame-pointer-g0-02 '-D_FORTIFY_SOURCE=1'-DNDEBUG -ffunction-sections. . . (remai g 69 argument (s) skipped)

Use--sandbox_debug to see verbose messages fron the sandbox mediapipe/calculators/tensor/image_to_tensor_converter_gl_buffer.cc: In function 'absl: :lts_2020 23: : StatusOr <std: :unique_ptr > mediapipe: : CreateImageToGLBuff nsorConverter (mediapipe: : CalculatorContext*,bool,mediapipe: : BorderMode) ': mediapipe/calculators/tensor/image_to_tensor_converter_gl_buffer.ck: 342: 10:error:could not con ＇result' from 'std: :unique_ptr <nediapipe: : {anonymous}::GlProcessor,std::default_delete<mediapi ｛anonymous} : : GLProcessor> > 「to 'absl: :lts_2020_09_23: : Status0r <std: :unique_ptr <mediapipe: : Image nsorConverter> > return result; Target//mediapipe/modules/face_detection:face_detection_short_range_cpu failed to build ICes to see the command Lines of failed build steps. ` How can I do next？ in jeton xavier ubuntu 18.04

Hello, have you solve this? I'm facing same problem with "return result;" in image_to_tensor_converter_gl_buffer.cc