search

googlesamples / mlkit

A collection of sample apps to demonstrate how to use Google's ML Kit APIs on Android and iOS
Apache License 2.0
3.5k stars 2.92k forks source link

Facedetection Static Bitmap image - Internal error occurred while executing MLKit tasks - Android #345

Closed archidvignesh closed 2 years ago

archidvignesh commented 3 years ago

So I am following Jetpack architecture and MVVM pattern. In the fragment, I use the image picker to fetch an image . If the user has successfully picked an image, then I call an isFaceDetected function from the viewModel. Here is the function:

//View Model
private fun detectFace(image: Bitmap) {
        val inputImage = InputImage.fromBitmap(image, 0)
        val detector = FaceDetection.getClient()
        val result = detector.process(inputImage).addOnFailureListener {
            Log.d("Authentication", it.toString())
        }.addOnSuccessListener {
            Log.d("Authentication", "Successful face detection")
        }
    }

Here is how I call this function from the fragment

@RequiresApi(Build.VERSION_CODES.P)
    private val getContentA = registerForActivityResult(ActivityResultContracts.StartActivityForResult()) { it ->
        val resultCode = it.resultCode
        val data = it.data

        if(resultCode == Activity.RESULT_OK) {
            val fileUri = data?.data!!
            val bitmap = when {
                Build.VERSION.SDK_INT < 28 -> MediaStore.Images.Media.getBitmap(
                    requireActivity().contentResolver,
                    fileUri
                )
                else -> {
                    val source = ImageDecoder.createSource(
                        requireActivity().contentResolver,
                        fileUri
                    )
                    ImageDecoder.decodeBitmap(source)
                }
            }

            viewModel.detectFace(bitmap)
        }
    }

I have followed all steps in documentation. I am using bundled model. I am getting error Internal error occurred while executing MLKit tasks. Here is the trace from logs

2021-08-23 17:21:24.116 464-1061/com.example.chemisstry I/DynamiteModule: Considering local module com.google.mlkit.dynamite.face:10000 and remote module com.google.mlkit.dynamite.face:0
2021-08-23 17:21:24.117 464-1061/com.example.chemisstry I/DynamiteModule: Selected local version of com.google.mlkit.dynamite.face
2021-08-23 17:21:24.117 464-767/com.example.chemisstry W/GooglePlayServicesUtil: Google Play services out of date for com.example.chemisstry.  Requires 203390000 but found 203019047
2021-08-23 17:21:24.118 464-767/com.example.chemisstry W/GoogleApiManager: The service for com.google.android.gms.common.internal.service.zap is not available: ConnectionResult{statusCode=SERVICE_VERSION_UPDATE_REQUIRED, resolution=null, message=null}
2021-08-23 17:21:24.156 464-1061/com.example.chemisstry W/LibraryVersion: Failed to get app version for libraryName: optional-module-face
2021-08-23 17:21:24.169 464-504/com.example.chemisstry I/mple.chemisstr: NativeAlloc concurrent copying GC freed 55198(2566KB) AllocSpace objects, 10(328KB) LOS objects, 62% free, 7514KB/19MB, paused 149us total 112.133ms
2021-08-23 17:21:24.193 464-768/com.example.chemisstry D/TransportRuntime.SQLiteEventStore: Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
2021-08-23 17:21:24.210 464-768/com.example.chemisstry D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
2021-08-23 17:21:24.228 464-768/com.example.chemisstry D/TransportRuntime.SQLiteEventStore: Storing event with priority=DEFAULT, name=FIREBASE_ML_SDK for destination cct
2021-08-23 17:21:24.230 464-767/com.example.chemisstry W/GooglePlayServicesUtil: Google Play services out of date for com.example.chemisstry.  Requires 203390000 but found 203019047
2021-08-23 17:21:24.230 464-767/com.example.chemisstry W/GoogleApiManager: The service for com.google.android.gms.common.internal.service.zap is not available: ConnectionResult{statusCode=SERVICE_VERSION_UPDATE_REQUIRED, resolution=null, message=null}
2021-08-23 17:21:24.233 464-1061/com.example.chemisstry V/FaceDetectorV2Jni: initialize.start()
2021-08-23 17:21:24.239 464-768/com.example.chemisstry D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, DEFAULT, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
2021-08-23 17:21:24.239 464-1061/com.example.chemisstry I/native: I0823 17:21:24.239522    1061 face_detector_v2_jni.cc:33] Loading models/fssd_25_8bit_v1.tflite
2021-08-23 17:21:24.240 464-1061/com.example.chemisstry I/native: I0823 17:21:24.240793    1061 face_detector_v2_jni.cc:33] Loading models/fssd_25_8bit_gray_v1.tflite
2021-08-23 17:21:24.242 464-1061/com.example.chemisstry I/native: I0823 17:21:24.242186    1061 face_detector_v2_jni.cc:33] Loading models/LMprec_600.emd
2021-08-23 17:21:24.257 464-768/com.example.chemisstry D/TransportRuntime.SQLiteEventStore: Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
2021-08-23 17:21:24.257 464-464/com.example.chemisstry I/InputTransport: Create ARC handle: 0x7476e027c0
2021-08-23 17:21:24.257 464-1061/com.example.chemisstry I/native: I0823 17:21:24.257549    1061 face_detector_v2_jni.cc:33] Loading models/BCLlefteyeclosed_200.emd
2021-08-23 17:21:24.257 464-1061/com.example.chemisstry I/native: I0823 17:21:24.257946    1061 face_detector_v2_jni.cc:33] Loading models/BCLrighteyeclosed_200.emd
2021-08-23 17:21:24.258 464-1061/com.example.chemisstry I/native: I0823 17:21:24.258188    1061 face_detector_v2_jni.cc:33] Loading models/BCLjoy_200.emd
2021-08-23 17:21:24.258 464-1061/com.example.chemisstry I/native: I0823 17:21:24.258554    1061 face_detector_v2_jni.cc:33] Loading models/MFT_fssd_fastgray.pb
2021-08-23 17:21:24.258 464-1061/com.example.chemisstry I/native: I0823 17:21:24.258692    1061 face_detector_v2_jni.cc:33] Loading models/contours.tfl
2021-08-23 17:21:24.265 464-787/com.example.chemisstry V/FA: Activity resumed, time: 2643730424
2021-08-23 17:21:24.269 464-464/com.example.chemisstry I/Choreographer: Skipped 30 frames!  The application may be doing too much work on its main thread.
2021-08-23 17:21:24.272 464-768/com.example.chemisstry D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
2021-08-23 17:21:24.273 464-1061/com.example.chemisstry I/native: I0823 17:21:24.273821    1061 face_detector_v2_jni.cc:33] Loading models/blazeface.tfl
2021-08-23 17:21:24.285 464-1061/com.example.chemisstry I/tflite: Initialized TensorFlow Lite runtime.
2021-08-23 17:21:24.295 464-768/com.example.chemisstry D/TransportRuntime.SQLiteEventStore: Storing event with priority=VERY_LOW, name=FIREBASE_ML_SDK for destination cct
2021-08-23 17:21:24.296 464-1061/com.example.chemisstry V/FaceDetectorV2Jni: initialize.end()
2021-08-23 17:21:24.308 464-768/com.example.chemisstry D/TransportRuntime.JobInfoScheduler: Upload for context TransportContext(cct, VERY_LOW, MSRodHRwczovL2ZpcmViYXNlbG9nZ2luZy5nb29nbGVhcGlzLmNvbS92MGNjL2xvZy9iYXRjaD9mb3JtYXQ9anNvbl9wcm90bzNc) is already scheduled. Returning...
2021-08-23 17:21:24.416 464-464/com.example.chemisstry D/ViewRootImpl[AuthenticationActivity]: msg2 update new insets InsetsState: {mDisplayFrame=Rect(0, 0 - 720, 1544), mSources= { InsetsSource: {mType=ITYPE_STATUS_BAR, mFrame=[0,0][720,56], mVisible=true}, InsetsSource: {mType=ITYPE_NAVIGATION_BAR, mFrame=[0,1460][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_TOP_GESTURES, mFrame=[0,0][720,56], mVisible=true}, InsetsSource: {mType=ITYPE_BOTTOM_GESTURES, mFrame=[0,1460][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_LEFT_GESTURES, mFrame=[0,0][0,1544], mVisible=true}, InsetsSource: {mType=ITYPE_RIGHT_GESTURES, mFrame=[720,0][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_TOP_TAPPABLE_ELEMENT, mFrame=[0,0][720,56], mVisible=true}, InsetsSource: {mType=ITYPE_BOTTOM_TAPPABLE_ELEMENT, mFrame=[0,1460][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_LEFT_DISPLAY_CUTOUT, mFrame=[0,0][-2147483648,1544], mVisible=true}, InsetsSource: {mType=ITYPE_TOP_DISPLAY_CUTOUT, mFrame=[0,0][720,54], mVisible=true}, InsetsSource: {mType=ITYPE_RIGHT_DISPLAY_CUTOUT, mFrame=[2147483647,0][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_BOTTOM_DISPLAY_CUTOUT, mFrame=[0,2147483647][720,1544], mVisible=true}, InsetsSource: {mType=ITYPE_IME, mFrame=[0,0][0,0], mVisible=false} }
2021-08-23 17:21:24.424 464-464/com.example.chemisstry D/Authentication: com.google.mlkit.common.MlKitException: Internal error has occurred when executing ML Kit tasks
2021-08-23 17:21:29.291 464-787/com.example.chemisstry V/FA: Inactivity, disconnecting from the service
2021-08-23 17:22:13.661 464-768/com.example.chemisstry D/TransportRuntime.CctTransportBackend: Making request to: ************************************************************************
2021-08-23 17:22:13.789 464-768/com.example.chemisstry D/Linux: [Posix_connect Debug]Process com.example.chemisstry :443 
2021-08-23 17:22:14.037 464-768/com.example.chemisstry I/TransportRuntime.CctTransportBackend: Status Code: 200
2021-08-23 17:22:14.038 464-768/com.example.chemisstry I/TransportRuntime.CctTransportBackend: Content-Type: application/json; charset=UTF-8
2021-08-23 17:22:14.038 464-768/com.example.chemisstry I/TransportRuntime.CctTransportBackend: Content-Encoding: gzip
archidvignesh commented 3 years ago

Here is another useful information:

If I try to do this with URI, it works fine. But bitmap is causing internal error.

This might be a useful tip as well. I was following the documentation step by step but I accidentally included both google play services module and the bundle module and built the gradle. Then I deleted the google plays services module as I need the model to be bundled to the app.

zhouyiself commented 3 years ago

From the log, the face engine is initialized successfully, so I'm wondering if there's something wrong with the input bitmap. Can you check if the bitmap is valid before feeding to ML Kit?

archidvignesh commented 3 years ago

Bitmap is ok. I upload to firebase storage as bitmap data and I’m able to get the url and set an image so I guess it’s fine.

SilverDestiny commented 2 years ago

Hi,

2021-08-23 17:21:24.424 464-464/com.example.chemisstry D/Authentication: com.google.mlkit.common.MlKitException: Internal error has occurred when executing ML Kit tasks

Could you print out the cause of this MlKitException as well? It'd be easier to check what failed if we have the full details. Thanks!

archidvignesh commented 2 years ago

@SilverDestiny Can you give me instructions on how I can get that for you? Sorry kind of new to android

SilverDestiny commented 2 years ago

MlKitException was constructed by the error message and cause Throwable by https://developer.android.com/reference/java/lang/Exception#Exception(java.lang.String,%20java.lang.Throwable)

From your code:

val result = detector.process(inputImage).addOnFailureListener {
            Log.d("Authentication", it.toString())
        }

I think the toString() here only prints the top level error message without underlying cause details. Probably you can log with Throwable https://developer.android.com/reference/android/util/Log#d(java.lang.String,%20java.lang.String,%20java.lang.Throwable) or just printStackTrace() https://developer.android.com/reference/java/lang/Throwable#printStackTrace()

jackqdyulei commented 2 years ago

Hey have you fixed the issue? Can we close this?

archidvignesh commented 2 years ago

Yes you can go ahead. Since our dev time was getting affected we ended up using URL. Like I said, works fine with url but bitmap was not working. You can try recreating by picking an image using image picker and supply that image to see if a face is detected. As you know, image picker gives us the bitmap. I was able to upload the bitmap to firebase, retrieve its url and load up the image using coil etc so data was not corrupted. Maybe you can try replicating this on your end. Sorry couldn’t assist further

On Wed, 29 Sep 2021 at 20:58, jackqdyulei @.***> wrote:

Hey have you fixed the issue? Can we close this?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/googlesamples/mlkit/issues/345#issuecomment-930285652, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSS4VQVSPVDZ4RC4KTGBB3UEMWARANCNFSM5CUOAYCA .

jackqdyulei commented 2 years ago

Hey could you at least give us following information?

1) SDK version com.google.mlkit:face-detection:

2) The Device API level where it breaks. (Since you added @RequiresAPI and have special code on API level < 28 )

archidvignesh commented 2 years ago

Sure,

  1. SDK ‘com.google.mlkit:face-detection:16.1.2’ SDK ‘com.google.mlkit:barcode-scanning:17.0.0’

  2. We are supporting devices that are only android 8.1 and above. Nothing less than that

  3. Phone used to test the feature real device - Vivo 1906 - android version

  4. Did not test this on other devices as we are limited on phone supply.

Let me know if you need any other information

On Wed, 29 Sep 2021 at 21:34, jackqdyulei @.***> wrote:

Hey could you at least give us following information?

1.

SDK version com.google.mlkit:face-detection: 2.

The Device API level where it breaks. (Since you added @RequiresAPI and have special code on API level < 28 )

1.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/googlesamples/mlkit/issues/345#issuecomment-930316655, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSS4VTTB3KKFSQNOPUAAI3UEM2HJANCNFSM5CUOAYCA .

jackqdyulei commented 2 years ago

Thanks for your report. The root cause is ImageDecoder.decodeBitmap(source) by default returns HARDWARE bitmap, which is not support by ML Kit.

We will consider whether to support it, or explicitly mention it in javadoc.

archidvignesh commented 2 years ago

Ok, thank you.

On Wed, 6 Oct 2021 at 23:49, jackqdyulei @.***> wrote:

Thanks for your report. The root cause is ImageDecoder.decodeBitmap(source) by default returns HARDWARE bitmap, which is not support by ML Kit.

We will consider whether to support it, or explicitly mention it in javadoc.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/googlesamples/mlkit/issues/345#issuecomment-936837995, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSS4VWWYVQEHR3XZVVPY63UFSHL7ANCNFSM5CUOAYCA .

jackqdyulei commented 2 years ago

Issue been fixed internally and will be available in future release.