Open qinweisen opened 2 months ago
Strangely, I was able to run cpu version successfully in 0.10.14 but still can't use gpu version cause graph has errors as follow:
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1724923248.305071 5186 demo_run_graph_main_gpu.cc:54] Get calculator graph config contents: # MediaPipe graph that performs multi-hand tracking with TensorFlow Lite on GPU.
# Used in the examples in
# mediapipe/examples/android/src/java/com/mediapipe/apps/handtrackinggpu.
# GPU image. (GpuBuffer)
input_stream: "input_video"
# GPU image. (GpuBuffer)
output_stream: "output_video"
# Collection of detected/predicted hands, each represented as a list of
# landmarks. (std::vector<NormalizedLandmarkList>)
output_stream: "hand_landmarks"
# Generates side packet cotaining max number of hands to detect/track.
node {
calculator: "ConstantSidePacketCalculator"
output_side_packet: "PACKET:num_hands"
node_options: {
[type.googleapis.com/mediapipe.ConstantSidePacketCalculatorOptions]: {
packet { int_value: 2 }
}
}
}
# Detects/tracks hand landmarks.
node {
calculator: "HandLandmarkTrackingGpu"
input_stream: "IMAGE:input_video"
input_side_packet: "NUM_HANDS:num_hands"
output_stream: "LANDMARKS:hand_landmarks"
output_stream: "HANDEDNESS:handedness"
output_stream: "PALM_DETECTIONS:palm_detections"
output_stream: "HAND_ROIS_FROM_LANDMARKS:hand_rects_from_landmarks"
output_stream: "HAND_ROIS_FROM_PALM_DETECTIONS:hand_rects_from_palm_detections"
}
# Subgraph that renders annotations and overlays them on top of the input
# images (see hand_renderer_gpu.pbtxt).
node {
calculator: "HandRendererSubgraph"
input_stream: "IMAGE:input_video"
input_stream: "DETECTIONS:palm_detections"
input_stream: "LANDMARKS:hand_landmarks"
input_stream: "HANDEDNESS:handedness"
input_stream: "NORM_RECTS:0:hand_rects_from_palm_detections"
input_stream: "NORM_RECTS:1:hand_rects_from_landmarks"
output_stream: "IMAGE:output_video"
}
I0000 00:00:1724923248.305955 5186 demo_run_graph_main_gpu.cc:60] Initialize the calculator graph.
I0000 00:00:1724923248.315212 5186 demo_run_graph_main_gpu.cc:64] Initialize the GPU.
MESA: error: ZINK: failed to choose pdev
libEGL warning: egl: failed to create dri2 screen
I0000 00:00:1724923249.335691 5186 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1724923249.352707 5224 gl_context.cc:357] GL version: 3.0 (OpenGL ES 3.0 Mesa 24.0.9-0ubuntu0.1), renderer: D3D12 (Intel(R) UHD Graphics 770)
I0000 00:00:1724923249.352851 5186 demo_run_graph_main_gpu.cc:70] Initialize the camera or load the video.
I0000 00:00:1724923249.363143 5186 demo_run_graph_main_gpu.cc:91] Start running the calculator graph.
I0000 00:00:1724923249.364231 5186 demo_run_graph_main_gpu.cc:96] Start grabbing and processing frames.
INFO: Initialized TensorFlow Lite runtime.
INFO: Created TensorFlow Lite delegate for GPU.
E0000 00:00:1724923249.397386 5186 demo_run_graph_main_gpu.cc:199] Failed to run the graph: Graph has errors:
Calculator::Open() for node "handlandmarktrackinggpu__palmdetectiongpu__inferencecalculator__handlandmarktrackinggpu__palmdetectiongpu__InferenceCalculator" failed: ; RET_CHECK failure (mediapipe/calculators/tensor/inference_calculator_gl.cc:194) (TfLiteGpuDelegateBindBufferToTensor( delegate_.get(), gpu_buffers_in_.back()->GetOpenGlBufferWriteView().name(), interpreter_->inputs()[i]))==(kTfLiteOk)
Strangely, I was able to run cpu version successfully in 0.10.14 but still can't use gpu version cause graph has errors as follow:
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1724923248.305071 5186 demo_run_graph_main_gpu.cc:54] Get calculator graph config contents: # MediaPipe graph that performs multi-hand tracking with TensorFlow Lite on GPU. # Used in the examples in # mediapipe/examples/android/src/java/com/mediapipe/apps/handtrackinggpu. # GPU image. (GpuBuffer) input_stream: "input_video" # GPU image. (GpuBuffer) output_stream: "output_video" # Collection of detected/predicted hands, each represented as a list of # landmarks. (std::vector<NormalizedLandmarkList>) output_stream: "hand_landmarks" # Generates side packet cotaining max number of hands to detect/track. node { calculator: "ConstantSidePacketCalculator" output_side_packet: "PACKET:num_hands" node_options: { [type.googleapis.com/mediapipe.ConstantSidePacketCalculatorOptions]: { packet { int_value: 2 } } } } # Detects/tracks hand landmarks. node { calculator: "HandLandmarkTrackingGpu" input_stream: "IMAGE:input_video" input_side_packet: "NUM_HANDS:num_hands" output_stream: "LANDMARKS:hand_landmarks" output_stream: "HANDEDNESS:handedness" output_stream: "PALM_DETECTIONS:palm_detections" output_stream: "HAND_ROIS_FROM_LANDMARKS:hand_rects_from_landmarks" output_stream: "HAND_ROIS_FROM_PALM_DETECTIONS:hand_rects_from_palm_detections" } # Subgraph that renders annotations and overlays them on top of the input # images (see hand_renderer_gpu.pbtxt). node { calculator: "HandRendererSubgraph" input_stream: "IMAGE:input_video" input_stream: "DETECTIONS:palm_detections" input_stream: "LANDMARKS:hand_landmarks" input_stream: "HANDEDNESS:handedness" input_stream: "NORM_RECTS:0:hand_rects_from_palm_detections" input_stream: "NORM_RECTS:1:hand_rects_from_landmarks" output_stream: "IMAGE:output_video" } I0000 00:00:1724923248.305955 5186 demo_run_graph_main_gpu.cc:60] Initialize the calculator graph. I0000 00:00:1724923248.315212 5186 demo_run_graph_main_gpu.cc:64] Initialize the GPU. MESA: error: ZINK: failed to choose pdev libEGL warning: egl: failed to create dri2 screen I0000 00:00:1724923249.335691 5186 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5 I0000 00:00:1724923249.352707 5224 gl_context.cc:357] GL version: 3.0 (OpenGL ES 3.0 Mesa 24.0.9-0ubuntu0.1), renderer: D3D12 (Intel(R) UHD Graphics 770) I0000 00:00:1724923249.352851 5186 demo_run_graph_main_gpu.cc:70] Initialize the camera or load the video. I0000 00:00:1724923249.363143 5186 demo_run_graph_main_gpu.cc:91] Start running the calculator graph. I0000 00:00:1724923249.364231 5186 demo_run_graph_main_gpu.cc:96] Start grabbing and processing frames. INFO: Initialized TensorFlow Lite runtime. INFO: Created TensorFlow Lite delegate for GPU. E0000 00:00:1724923249.397386 5186 demo_run_graph_main_gpu.cc:199] Failed to run the graph: Graph has errors: Calculator::Open() for node "handlandmarktrackinggpu__palmdetectiongpu__inferencecalculator__handlandmarktrackinggpu__palmdetectiongpu__InferenceCalculator" failed: ; RET_CHECK failure (mediapipe/calculators/tensor/inference_calculator_gl.cc:194) (TfLiteGpuDelegateBindBufferToTensor( delegate_.get(), gpu_buffers_in_.back()->GetOpenGlBufferWriteView().name(), interpreter_->inputs()[i]))==(kTfLiteOk)
I've fixed it by changing the OpenGL default GPU device with export MESA_D3D12_DEFAULT_ADAPTER_NAME=NVIDIA
, after that did ES version meet the requirements of 3.1.
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/generated_message_reflection.cc:2933] CHECK failed: file != nullptr:
terminate called after throwing an instance of 'google::protobuf::FatalException'
what(): CHECK failed: file != nullptr:
Aborted
I didn't do anything and the problem disappeared, but when I tried to build a dynamic link library, it reappeared. It seems that the error is thrown by the following code:
mediapipe::CalculatorGraphConfig config =
mediapipe::ParseTextProtoOrDie<mediapipe::CalculatorGraphConfig>(calculator_graph_config_contents);
and nullptr caused by loading google/protobuf/descriptor.proto
.
I really want to know what caused this problem.
@qinweisen I enountered the same issue https://github.com/google-ai-edge/mediapipe/issues/5655#issue-2554015725, on different platform though. Have you solved the problem?
OS Platform and Distribution
Linux Ubuntu 24.04
Compiler version
gcc version 13.2.0 (Ubuntu 13.2.0-23ubuntu4)
Programming Language and version
C++ 17
Installed using virtualenv? pip? Conda?(if python)
No response
MediaPipe version
0.10.15
Bazel version
6.1.1
XCode and Tulsi versions (if iOS)
No response
Android SDK and NDK versions (if android)
No response
Android AAR (if android)
None
OpenCV version (if running on desktop)
4.6.0
Describe the problem
I successfully build hand_tracking_gpu, but protobuf threw a fatal error during runtime.
I built hand_tracking_gpu with the following command:
$ bazel build -c opt --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 mediapipe/examples/desktop/hand_tracking:hand_tracking_gpu
And it returned "INFO: Build completed successfully, 3370 total actions"
But while I tried to run with the following command:
$ GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/hand_tracking/hand_tracking_gpu --calculator_graph_config_file=mediapipe/graphs/hand_tracking/hand_tracking_desktop_live_gpu.pbtxt --input_video_path=input.mp4 --output_video_path=output.mp4
Libprotobuf threw a fatal error, complete logs are as follows.
By the way, I'v already download ".tflite" files and put them in: "mediapipe/modules/hand_landmark" and "mediapipe/modules/palm_detection".
Complete Logs