Effect Rendering in Python3

[shreshtashetty]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

No

OS Platform and Distribution

Linux Ubuntu 18.04

MediaPipe Tasks SDK version

No response

Task name (e.g. Image classification, Gesture recognition etc.)

Effect Rendering

Programming Language and version (e.g. C++, Python, Java)

Python

Describe the actual behavior

-

Describe the expected behaviour

I have looked at the face landmarker task guide which gives face blendshapes, but how do I use them for effect rendering in Python3? Is there any way to use effect_renderer_calculator.cc in Python3? I have looked at #1155 and other issues which perform rendering in Java, I'm guessing.

Is https://learnopencv.com/create-snapchat-instagram-filters-using-mediapipe the only way to do it in Python3?

I'm a bit new to this, so I'm sorry if this query sounds too basic.

[schmidt-sebastian]

We currently do not offer a high level API for effect rendering or graph customization. You can however run and create your own graphs via the underlying framework. This requires quite a bit of setup though: https://developers.google.com/mediapipe/framework

There are examples in our code base for how to run custom graphs: https://github.com/google/mediapipe/blob/master/mediapipe/python/solutions/face_mesh.py

It is probably easier to build your own rendering code. For that, you can also look at https://github.com/google/mediapipe/blob/master/mediapipe/tasks/web/vision/core/drawing_utils.ts

We hope that we can provide an easier way to customized graphs in the future.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[shreshtashetty]

@schmidt-sebastian, is it easier to do this in C++ (using the faceeffect graph) than in Python?