search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://mediapipe.dev
Apache License 2.0
26.71k stars 5.08k forks source link

MediaPipe Object Detection crashes on Android emulators if API > 30 #5134

Open BoHellgren opened 6 months ago

BoHellgren commented 6 months ago

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Android 11, 12, 13

Mobile device if the issue happens on mobile device

Emulated Pixel 5

Browser and version if the issue happens on browser

No response

Programming Language and version

Kotlin

MediaPipe version

implementation 'com.google.mediapipe:tasks-vision:latest.release'

Bazel version

No response

Solution

Object Detection

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

Fatal exception if API level > 30

Describe the expected behaviour

To work as it does on API level 30

Standalone code/steps you may have used to try to get what you need

D/AndroidRuntime( 7815): Shutting down VM
E/AndroidRuntime( 7815): FATAL EXCEPTION: main
E/AndroidRuntime( 7815): Process: se.ndssoft.mycamx_example, PID: 7815
E/AndroidRuntime( 7815): java.lang.UnsatisfiedLinkError: dlopen failed: library "libmediapipe_tasks_vision_jni.so" not found
E/AndroidRuntime( 7815):        at java.lang.Runtime.loadLibrary0(Runtime.java:1077)
E/AndroidRuntime( 7815):        at java.lang.Runtime.loadLibrary0(Runtime.java:998)
E/AndroidRuntime( 7815):        at java.lang.System.loadLibrary(System.java:1656)
E/AndroidRuntime( 7815):        at com.google.mediapipe.tasks.vision.core.BaseVisionTaskApi.<clinit>(BaseVisionTaskApi.java:37)
E/AndroidRuntime( 7815):        at com.google.mediapipe.tasks.vision.objectdetector.ObjectDetector.createFromOptions(ObjectDetector.java:174)
E/AndroidRuntime( 7815):        at se.ndssoft.mycamx.MycamxPlugin.startCamera$lambda-5(MycamxPlugin.kt:168)

MycamxPlugin.kt:168 follows
objectDetector = ObjectDetector.createFromOptions(mContext, options)

Other info / Complete Logs

My app works on a real Samsung Galaxy S10e, an emulated Pixel 5 API 30 and an emulated Nexus 5 API 30. But it crashes on Pixel 5 API 31 and Nexus 5 API 33 with the message dlopen failed: library "libmediapipe_tasks_vision_jni.so" not found
BoHellgren commented 6 months ago

The medipipe/examples/objectdetection app also crashes when run on an emulator with API > 30. It seems that x86 images are OK, but x86_64 images are not. Then this app crashes because libmediapipe_tasks_vision_jni.so cannot be found. This crash comes at CameraFragment.kt:135

kuaashish commented 6 months ago

Hi @schmidt-sebastian,

It appears to be similar to the issue https://github.com/google/mediapipe/issues/5100, which has already been reported and is causing crashes on Android emulators. Could you please investigate this issue as well?

Thank you!!!

BoHellgren commented 6 months ago

I downloaded your sample program medipipe/examples/objectdetection and made only two small changes, to app build.gradle:

BoHellgren commented 6 months ago

If I look at the APK file, both debug and release, there is a libmediapipe_tasks_vision_jni.so for x86, arm64-v8a, and arme-abiv7a, but not for x86_64. Have you for some reason not compiled MediaPipe for this architecture??

This bug stops me from releasing my new app since I cannot test it properly. I would be grateful if you fix this asap.

BoHellgren commented 6 months ago

f I look at the APK file, both debug and release, there is a libmediapipe_tasks_vision_jni.so for x86, arm64-v8a, and arme-abiv7a, but not for x86_64. Have you for some reason not compiled MediaPipe for this architecture??

This bug stops me from releasing my new app since I cannot test it properly. I would be grateful if you fix this asap.

Den mån 12 feb. 2024 kl 09:36 skrev kuaashish @.***>:

Hi @schmidt-sebastian https://github.com/schmidt-sebastian,

It appears to be similar to the issue #5100 https://github.com/google/mediapipe/issues/5100, which has already been reported and is causing crashes on Android emulators. Could you please investigate this issue as well?

Thank you!!!

— Reply to this email directly, view it on GitHub https://github.com/google/mediapipe/issues/5134#issuecomment-1938235840, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEN5JJLXELYQBNRYKM46OOTYTHIBJAVCNFSM6AAAAABDC2CDX2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZYGIZTKOBUGA . You are receiving this because you authored the thread.Message ID: @.***>

BoHellgren commented 6 months ago

@schmidt-sebastian When can you look at this?

kuaashish commented 6 months ago

Hi @BoHellgren,

We have identified that using an emulator to run this program might not be ideal due to hardware limitations and the program's reliance on specific hardware features for optimal performance. Using a physical device is highly recommended for the best experience and functionality.

Because, Machine learning programs often require direct interaction with real-world hardware resources to function effectively. Emulators, while valuable for development and testing, may not provide the necessary hardware capabilities for this program to run smoothly. Unfortunately, due to limitations, We can not do much about this issue.

Thank you!!

BoHellgren commented 6 months ago

I agree that testing ML apps in an emulator has its problems. For example, to test your Object Detection sample program, one has to use Extended Controls to mount an image on a wall in the emulated apartment, and then use mouse and QWERASDF keys to move the emulated camera to face the wall with the photo. This does not mean that testing on emulators is useless. On the contrary, unless you have a lot of physical devices, emulators are the only way to test that your app runs on all API levels. Testing on Firebase Test Lab is not an alternative for ML apps, since you then don't have any control of the camera whatsoever.

I am convinced that you can make MediaPipe run on emulators if you want to, and I urge you to fix this. I think many developers will say no thanks to MediaPipe if they hear that you only support testing on real devices.

otro678 commented 5 months ago

Unfortunately, due to limitations, We can not do much about this issue.

@kuaashish When added via gradle mediapipe literally provides x86 library and does not provide x86_64.

It would crash on a real hardware x64 device. This crash is not due emulator limitations, this is due the missing jni library, lol.

Have you even read the issue, or did you just generated your response using ChatGPT?

kuaashish commented 5 months ago

Hi @schmidt-sebastian,

Could you please have look into this issue?

Thank you!!

Hassaan69 commented 4 months ago

Did anyone find any solution to this problem ?

I have tried hand gesture example on s23 ultra running on android 14 and I get the same error basically any device having SDK > 30 gives same error i.e jni file not found.

aartikov commented 3 months ago

@kuaashish @schmidt-sebastian There are real devices with x86_64 architecture, such as the ASUS ZenFone Series and Lenovo K900. Do you have plans to support these devices?

Additionally, the absence of libmediapipe_tasks_vision_jni.so for x86_64 can result in rejection when publishing to the Google Play Store. What solutions do you suggest for this issue?