Error in Finding Runtime Files for Hand Tracking on Windows 11 with MediaPipe Example

hans-brgs commented 1 year ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

No

OS Platform and Distribution

Windows 11

Mobile device if the issue happens on mobile device

No response

Browser and version if the issue happens on browser

No response

Programming Language and version

C++

MediaPipe version

V0.10.3

Bazel version

6.1.1

Solution

All examples requiring a runtime file

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

While attempting to run the hand tracking example (or other example) on Windows 11, the build succeeds without any issues, but the execution fails with an error related to file paths.

Describe the expected behaviour

When I encountered the error, I began an investigation into the code that determined the path to the resource files required for execution.

In the compilation file located at "mediapipe_release\mediapipe\util\BUILD", I identified a platform-specific implementation for Windows, defined in the "resource_util_windows.cc" file.

To address the issue, I modified the compilation configuration to utilize the default resource utility file for Windows builds, making the change as follows:

"//mediapipe:windows": ["resource_util_default.cc"],

This change resolving the error. So, it appears that the implementation of resource_util_windows.cc may need to be revised or adjusted for compatibility with the current setup on Windows 11.

Standalone code/steps you may have used to try to get what you need

Build Details

Command: bazel build --config=windows //mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu
.bazelrc Configuration:

# Tensorflow needs remote repo
common --experimental_repo_remote_exec

# Basic build settings
build --jobs 128
build --define='absl=1'  # for gtest
build --enable_platform_specific_config
build -c opt

# windows-specific options
build:windows --cxxopt=/std:c++17
build:windows --host_cxxopt=/std:c++17
build:windows --copt=/w                                             
build:windows --copt=/DBOOST_ERROR_CODE_HEADER_ONLY                     
build:windows --copt=/DMESA_EGL_NO_X11_HEADERS                          
build:windows --copt=/DEGL_NO_X11                                       
build:windows --copt=/D_USE_MATH_DEFINES                                
build:windows --host_copt=/D_USE_MATH_DEFINES
build:windows --define MEDIAPIPE_DISABLE_GPU=1

Command to run :

bazel-bin\mediapipe\examples\desktop\hand_tracking\hand_tracking_cpu --calculator_graph_config_file=mediapipe\graphs\hand_tracking\hand_tracking_desktop_live.pbtxt

Other info / Complete Logs

**logs :**

I20230814 09:37:56.911255 18272 demo_run_graph_main.cc:49] Get calculator graph config contents: # MediaPipe graph that performs hands tracking on desktop with TensorFlow
# Lite on CPU.
# Used in the example in
# mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu.

# CPU image. (ImageFrame)
input_stream: "input_video"

# CPU image. (ImageFrame)
output_stream: "output_video"

# Generates side packet cotaining max number of hands to detect/track.
node {
  calculator: "ConstantSidePacketCalculator"
  output_side_packet: "PACKET:num_hands"
  node_options: {
    [type.googleapis.com/mediapipe.ConstantSidePacketCalculatorOptions]: {
      packet { int_value: 2 }
    }
  }
}

# Detects/tracks hand landmarks.
node {
  calculator: "HandLandmarkTrackingCpu"
  input_stream: "IMAGE:input_video"
  input_side_packet: "NUM_HANDS:num_hands"
  output_stream: "LANDMARKS:landmarks"
  output_stream: "HANDEDNESS:handedness"
  output_stream: "PALM_DETECTIONS:multi_palm_detections"
  output_stream: "HAND_ROIS_FROM_LANDMARKS:multi_hand_rects"
  output_stream: "HAND_ROIS_FROM_PALM_DETECTIONS:multi_palm_rects"
}

# Subgraph that renders annotations and overlays them on top of the input
# images (see hand_renderer_cpu.pbtxt).
node {
  calculator: "HandRendererSubgraph"
  input_stream: "IMAGE:input_video"
  input_stream: "DETECTIONS:multi_palm_detections"
  input_stream: "LANDMARKS:landmarks"
  input_stream: "HANDEDNESS:handedness"
  input_stream: "NORM_RECTS:0:multi_palm_rects"
  input_stream: "NORM_RECTS:1:multi_hand_rects"
  output_stream: "IMAGE:output_video"
}
I20230814 09:37:56.916254 18272 demo_run_graph_main.cc:55] Initialize the calculator graph.
I20230814 09:37:56.921777 18272 demo_run_graph_main.cc:59] Initialize the camera or load the video.
I20230814 09:38:24.554293 18272 demo_run_graph_main.cc:80] Start running the calculator graph.
I20230814 09:38:24.555294 18272 demo_run_graph_main.cc:85] Start grabbing and processing frames.
bazel-bin/mediapipe/mediapipe/modules/palm_detection/palm_detection_full.tflitebazel-bin/mediapipe/mediapipe/modules/hand_landmark/hand_landmark_full.tflite 

E20230814 09:38:25.118510 18272 demo_run_graph_main.cc:155] Failed to run the graph: Graph has errors: 
Calculator::Open() for node "handlandmarktrackingcpu__handlandmarkcpu__handlandmarkmodelloader__LocalFileContentsCalculator" failed: ; Can't find file: mediapipe/modules/hand_landmark/hand_landmark_full.tflite
Calculator::Open() for node "handlandmarktrackingcpu__palmdetectioncpu__palmdetectionmodelloader__LocalFileContentsCalculator" failed: ; Can't find file: mediapipe/modules/palm_detection/palm_detection_full.tflite

kuaashish commented 1 year ago

Hello @joezoug,

Could you please look into this issue. Thank you

schmidt-sebastian commented 9 months ago

Did you try setting the runtime dir (https://source.corp.google.com/piper///depot/google3/third_party/mediapipe/util/resource_util_windows.cc;rcl=526658482;l=26) ?

If you don't do that, you will need to include the files as runtime dependencies via your Build as Bazel uses a runtime map to look up the locations of any file that it is loading on Windows.

google-ai-edge / mediapipe