Make android solutions(face detection) working with 'minifyEnabled true' build option.

[farmaker47]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

No

OS Platform and Distribution

Android 13

Mobile device if the issue happens on mobile device

All devices

Browser and version if the issue happens on browser

No response

Programming Language and version

Kotlin 1.9.23

MediaPipe version

0.10.14

Bazel version

No response

Solution

Face Detector

Android Studio, NDK, SDK versions (if issue is related to building in Android environment)

No response

Xcode & Tulsi version (if issue is related to building for iOS)

No response

Describe the actual behavior

Project builds with minifyEnable true but Face detector fails to load model.

Describe the expected behaviour

Builds, loads and uses model with minifyEnable = true

Standalone code/steps you may have used to try to get what you need

The project I use is under NDA and in production and unfortunately I cannot share a link. Steps are easy. Using FaceDetector simply change minifyEnable of the release version to true. Solutions like this one: https://github.com/google-ai-edge/mediapipe/issues/3509 do not work anymore. As a matter of fact the issue is not building but making it work.

Placing some logs I see that the initialization procedure fails as a runtimeexception here:

try {
            val optionsBuilder =
                FaceDetector.FaceDetectorOptions.builder()
                    .setBaseOptions(baseOptionsBuilder.build())
                    .setMinDetectionConfidence(threshold)
                    .setRunningMode(runningMode)

            when (runningMode) {
                RunningMode.IMAGE,
                RunningMode.VIDEO -> optionsBuilder.setRunningMode(runningMode)

                RunningMode.LIVE_STREAM ->
                    optionsBuilder.setRunningMode(runningMode)
                        .setResultListener(this::returnLivestreamResult)
                        .setErrorListener(this::returnLivestreamError)
            }

            val options = optionsBuilder.build()
            detector = FaceDetector.createFromOptions(context, options)
            Log.v(TAG, "Detector initialized.")
        } catch (e: IllegalStateException) {
            detectorListener?.onError(
                "Face detector failed to initialize. See error logs for details"
            )
            Log.v(TAG, "TFLite failed to load model with error1: " + e.message)
        } catch (e: RuntimeException) {
            detectorListener?.onError(
                "Face detector failed to initialize. See error logs for " +
                        "details", GPU_ERROR
            )
            Log.v(
                TAG,
                "Face detector failed to load model with error2: " + e.message
            )
        }

the proguard rules I use are:

-keep class javax.lang.model.** { *; }
-keep class autovalue.shaded.com.squareup.javapoet.** { *; }
-keep class okhttp3.internal.platform.** { *; }

-keep class com.google.mediapipe.solutioncore.** { *; }
-keep class com.google.protobuf.** { *; }

-dontwarn com.google.protobuf.**
-dontwarn javax.lang.model.**
-dontwarn okhttp3.internal.platform.**

Other info / Complete Logs

The logs are:

Face detector failed to load model with error2: Field platform_ for w4.K not found. Known fields are [public static final w4.K w4.K.W, public static volatile com.google.protobuf.GeneratedMessageLite$DefaultInstanceBasedParser w4.K.X]

[erfansn]

I have this issue too

[erfansn]

These three rules worked for me:

-keep class com.google.mediapipe.proto.** { *; }
-keepclassmembers class * extends com.google.protobuf.GeneratedMessageLite { *; }
-keep class com.google.common.flogger.** { *; }

[farmaker47]

I can verify that the solution @erfansn provided also works in my project but I think this is a workaround rather than a solution for MediaPipe Facedetector task. I hope this issue stays open until they have a version that we do not have to apply rules in proguard file.

[kuaashish]

Hi @farmaker47,

We will keep the issue open until it is resolved. Additionally, we are sharing this with our team to ensure it remains on our radar. However, we cannot provide an exact timeline for the fix.

Thank you!!