BTLDXX-x
commented
1 year ago my face_mesh_smooth.pbtxt:
# MediaPipe graph that performs face mesh with TensorFlow Lite on GPU.
# GPU buffer. (GpuBuffer)
input_stream: "input_video"
# Max number of faces to detect/process. (int)
input_side_packet: "num_faces"
# Output image with rendered results. (GpuBuffer)
output_stream: "output_video"
# Collection of detected/processed faces, each represented as a list of
# landmarks. (std::vector<NormalizedLandmarkList>)
output_stream: "smoothed_face_landmarks"
# Throttles the images flowing downstream for flow control. It passes through
# the very first incoming image unaltered, and waits for downstream nodes
# (calculators and subgraphs) in the graph to finish their tasks before it
# passes through another image. All images that come in while waiting are
# dropped, limiting the number of in-flight images in most part of the graph to
# 1. This prevents the downstream nodes from queuing up incoming images and data
# excessively, which leads to increased latency and memory usage, unwanted in
# real-time mobile applications. It also eliminates unnecessarily computation,
# e.g., the output produced by a node may get dropped downstream if the
# subsequent nodes are still busy processing previous inputs.
node {
calculator: "FlowLimiterCalculator"
input_stream: "input_video"
input_stream: "FINISHED:output_video"
input_stream_info: {
tag_index: "FINISHED"
back_edge: true
}
output_stream: "throttled_input_video"
}
# Defines side packets for further use in the graph.
node {
calculator: "ConstantSidePacketCalculator"
output_side_packet: "PACKET:with_attention"
node_options: {
[type.googleapis.com/mediapipe.ConstantSidePacketCalculatorOptions]: {
packet { bool_value: true }
}
}
}
# Subgraph that detects faces and corresponding landmarks.
node {
calculator: "FaceLandmarkFrontGpu"
input_stream: "IMAGE:throttled_input_video"
input_side_packet: "NUM_FACES:num_faces"
input_side_packet: "WITH_ATTENTION:with_attention"
output_stream: "LANDMARKS:multi_face_landmarks"
output_stream: "ROIS_FROM_LANDMARKS:face_rects_from_landmarks"
output_stream: "DETECTIONS:face_detections"
output_stream: "ROIS_FROM_DETECTIONS:face_rects_from_detections"
}
# Extracts a single set of face landmarks associated with the most prominent
# face detected from a collection.
node {
calculator: "SplitNormalizedLandmarkListVectorCalculator"
input_stream: "multi_face_landmarks"
output_stream: "face_landmarks"
node_options: {
[type.googleapis.com/mediapipe.SplitVectorCalculatorOptions] {
ranges: { begin: 0 end: 1 }
element_only: true
}
}
}
# Extracts the input image frame dimensions as a separate packet.
node {
calculator: "ImagePropertiesCalculator"
input_stream: "IMAGE:throttled_input_video"
output_stream: "SIZE:input_image_size"
}
# Applies smoothing to the single set of face landmarks.
node {
calculator: "FaceLandmarksSmoothing"
input_stream: "NORM_LANDMARKS:face_landmarks"
input_stream: "IMAGE_SIZE:input_image_size"
output_stream: "NORM_FILTERED_LANDMARKS:smoothed_face_landmarks"
}
# Subgraph that renders face-landmark annotation onto the input image.
node {
calculator: "FaceRendererGpu"
input_stream: "IMAGE:throttled_input_video"
input_stream: "LANDMARKS:smoothed_face_landmarks"
input_stream: "NORM_RECTS:face_rects_from_landmarks"
input_stream: "DETECTIONS:face_detections"
output_stream: "IMAGE:output_video"
}the BUILD file:
load(
"//mediapipe/framework/tool:mediapipe_graph.bzl",
"mediapipe_binary_graph",
)
licenses(["notice"])
package(default_visibility = ["//visibility:public"])
cc_library(
name = "mobile_calculators",
deps = [
"//mediapipe/calculators/core:flow_limiter_calculator",
"//mediapipe/graphs/face_mesh_smooth/subgraphs:face_renderer_gpu",
"//mediapipe/graphs/face_mesh_smooth/subgraphs:face_landmarks_smoothing",
"//mediapipe/modules/face_landmark:face_landmark_front_gpu",
],
)
mediapipe_binary_graph(
name = "face_mesh_mobile_gpu_binary_graph",
graph = "face_mesh_smooth.pbtxt",
output_name = "face_mesh_mobile_gpu.binarypb",
deps = [":mobile_calculators"],
)i followed the doc to build the aar. But i met a problem:
unknown: Input stream "throttled_input_video" of calculator "facelandmarkfrontgpu__ImagePropertiesCalculator_2" expects packets of type "mediapipe::GpuBuffer"
but the connected output stream will contain packets of type "[Same Type As OneOf<::mediapipe::Image, mediapipe::ImageFrame>]"what should i do to solve this problem? thanks very much
ayushgdev
google-ml-butler[bot]
Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
None
OS Platform and Distribution
android
MediaPipe version
latest
Bazel version
5.2.0
Solution
FaceMesh
Programming Language and version
java
Describe the actual behavior
Describe the expected behaviour
No response
Standalone code/steps you may have used to try to get what you need
No response
Other info / Complete Logs
No response