google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.31k stars 5.13k forks source link

Is it possible to pass in a transparent stream? #4352

Closed Emma-He closed 1 year ago

Emma-He commented 1 year ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

No response

MediaPipe version

No response

Bazel version

No response

Solution

No response

Programming Language and version

No response

Describe the actual behavior

I am trying to use the MaskOverlay calculator and the SelfieSegmentation module to create a segmented person with transparent background.

Describe the expected behaviour

No response

Standalone code/steps you may have used to try to get what you need

Currently I have achieved something similar by doing 

node {
  calculator: "MaskOverlayCalculator"
  input_stream: "VIDEO:0:segmentation_mask"
  input_stream: "VIDEO:1:throttled_input_video"
  input_stream: "MASK:segmentation_mask"
  output_stream: "OUTPUT:output_video"
}

However, this creates a red outline around the segmented person. So I was wondering if I can pass in a transparent/empty stream as an input or somehow change the segmentation mask from red to transparent.

Other info / Complete Logs

No response

ayushgdev commented 1 year ago

Hello @Emma-He Would you please provide details on the following:

Emma-He commented 1 year ago

Hello @ayushgdev, thank you for getting back to me. To answer your questions:

  1. I am building selfie_segmentation_gpu
  2. iOS version 16.3.1
  3. I believe it is the newest version?
  4. This is the graph I am currently using

input_stream: "input_video"

output_stream: "output_video"

node { calculator: "FlowLimiterCalculator" input_stream: "input_video" input_stream: "FINISHED:output_video" input_stream_info: { tag_index: "FINISHED" back_edge: true } output_stream: "throttled_input_video" }

node { calculator: "SelfieSegmentationGpu" input_stream: "IMAGE:throttled_input_video" output_stream: "SEGMENTATION_MASK:segmentation_mask" }

node { calculator: "MaskOverlayCalculator" input_stream: "VIDEO:0:segmentation_mask" input_stream: "VIDEO:1:throttled_input_video" input_stream: "MASK:segmentation_mask" output_stream: "OUTPUT:segmented_output" }

node { calculator: "MaskOverlayCalculator" input_stream: "VIDEO:0:segmented_output" input_stream: "VIDEO:1:throttled_input_video" input_stream: "MASK:segmentation_mask" output_stream: "OUTPUT:output_video" }

Emma-He commented 1 year ago

Hello @ayushgdev , I think I found an alternative solution, thank you for your time!