google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.74k stars 5.18k forks source link

Cannot initialize an example CalculatorGraph to use it in Python #4682

Open BastinFlorian opened 1 year ago

BastinFlorian commented 1 year ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

No

OS Platform and Distribution

Mac M1 Ventura

Mobile device if the issue happens on mobile device

None

Browser and version if the issue happens on browser

None

Programming Language and version

Python

MediaPipe version

0.10.3

Bazel version

6.1.1

Solution

Hand Tracking

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

None

Xcode & Tulsi version (if issue is related to building for iOS)

None

Describe the actual behavior

I tried to initialize a random CalculatorGraph followoing the option 1 with this 
[binary file copied as text](https://github.com/google/mediapipe/blob/master/mediapipe/graphs/hand_tracking/hand_tracking_desktop_live.pbtxt)
following  [thetutorial](https://developers.google.com/mediapipe/framework/getting_started/python_framework#graph) 

I got the following error 
`RuntimeError: ValidatedGraphConfig Initialization failed.
No registered object with name: HandRendererSubgraph; Unable to find Calculator "HandRendererSubgraph"`

### Describe the expected behaviour

```shell
I would like to initialize a graph and use is in Python

Standalone code/steps you may have used to try to get what you need

config_text = """
# MediaPipe graph that performs hands tracking on desktop with TensorFlow
# Lite on CPU.
# Used in the example in
# mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu.

# CPU image. (ImageFrame)
input_stream: "input_video"

# CPU image. (ImageFrame)
output_stream: "output_video"

# Generates side packet cotaining max number of hands to detect/track.
node {
  calculator: "ConstantSidePacketCalculator"
  output_side_packet: "PACKET:num_hands"
  node_options: {
    [type.googleapis.com/mediapipe.ConstantSidePacketCalculatorOptions]: {
      packet { int_value: 2 }
    }
  }
}

# Detects/tracks hand landmarks.
node {
  calculator: "HandLandmarkTrackingCpu"
  input_stream: "IMAGE:input_video"
  input_side_packet: "NUM_HANDS:num_hands"
  output_stream: "LANDMARKS:landmarks"
  output_stream: "HANDEDNESS:handedness"
  output_stream: "PALM_DETECTIONS:multi_palm_detections"
  output_stream: "HAND_ROIS_FROM_LANDMARKS:multi_hand_rects"
  output_stream: "HAND_ROIS_FROM_PALM_DETECTIONS:multi_palm_rects"
}

# Subgraph that renders annotations and overlays them on top of the input
# images (see hand_renderer_cpu.pbtxt).
node {
  calculator: "HandRendererSubgraph"
  input_stream: "IMAGE:input_video"
  input_stream: "DETECTIONS:multi_palm_detections"
  input_stream: "LANDMARKS:landmarks"
  input_stream: "HANDEDNESS:handedness"
  input_stream: "NORM_RECTS:0:multi_palm_rects"
  input_stream: "NORM_RECTS:1:multi_hand_rects"
  output_stream: "IMAGE:output_video"
}

"""

graph = mp.CalculatorGraph(graph_config=config_text)
output_packets = []
graph.observe_output_stream(
    'out_stream',
    lambda stream_name, packet:
        output_packets.append(mp.packet_getter.get_str(packet)))
print(graph)

Other info / Complete Logs

Using the binary file does not work. 
I have the following error 

NameError: name 'resources' is not defined
BastinFlorian commented 1 year ago

Any news ? @kuaashish