search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://mediapipe.dev
Apache License 2.0
26.29k stars 5.05k forks source link

Can't use MediaPipe in VisionOS development #5336

Open boroicamarius opened 3 months ago

boroicamarius commented 3 months ago

OS Platform and Distribution

VisionOS 1.0/1.1

Compiler version

No response

Programming Language and version

Swift

Installed using virtualenv? pip? Conda?(if python)

No response

MediaPipe version

10.12

Bazel version

No response

XCode and Tulsi versions (if iOS)

15.3

Android SDK and NDK versions (if android)

No response

Android AAR (if android)

None

OpenCV version (if running on desktop)

No response

Describe the problem

We want to use MediaPipe on visionOS, we just can't, I've tried with cocoapods, imports do not work if the projects are from different platforms. And I don't know how to build it for visionOS using bazel.

Complete Logs

It just doesn't work.
kuaashish commented 2 months ago

Hi @boroicamarius,

Thank you for bringing this to our attention. It appears that we have not yet tested this scenario, and its implementation as a new feature will depend on demand. To expedite the process, could you please provide some specific use cases in VisionOS that would be helpful for us to consider when implementing this feature?

boroicamarius commented 2 months ago

We would like to extract facial features from the persona that the vision pro gives us, we only have access to a 2d projection of it and we need accuracy to understand how the face moves. Other than that, if there is no demand but we would like to implement it on the vision pro, could you give us a scheme to understand how the code works and how we could create a new build target using bazel?

Razvi103 commented 2 months ago

I'm also trying to use MediaPipe on VisionOs for one of my projects. It would be very helpful if the model could extract facial features from the Persona.

kuaashish commented 2 months ago

Hi @boroicamarius, @Razvi103,

Thank you for the thorough explanation. Integrating MediaPipe into VisionPro seems like an interesting feature request. We will mark it as feature request for further consideration and bring it to the attention of our team internally.

Thank you!!