[Feature Request] ML Kit: On-Device Model Download API

[JvmName]

I'd like to request an API to query the download status for MLKit's on-device models. Ideally, this API would allow the following:

1. return the current status of the on-device model for a given "kit" (e.g., barcode, face, etc.). Potential responses could be UNAVAILABLE, AVAILABLE, DOWNLOADING, etc.
2. initiate a download for a specified "kit" and ideally allow an observer to be attached to monitor completion (e.g., Task<Void)
3. (optional) delete a downloaded model

Here's the current documentation with regards to on-device models:

Optional but recommended: Configure your app to automatically download the ML model to the device after your app is installed from the Play Store. To do so, add the following declaration to your app's AndroidManifest.xml file:

<application ...>
 ...
 <meta-data
     android:name="com.google.firebase.ml.vision.DEPENDENCIES"
     android:value="barcode" />
 <!-- To use multiple models: android:value="barcode,model2,model3" -->
</application>

If you do not enable install-time model downloads, the model will be downloaded the first time you run the detector. Requests you make before the download has completed will produce no results.

(from https://firebase.google.com/docs/ml-kit/android/read-barcodes, emphasis mine)

I think it's crucial to have APIs to monitor the state of the on-device models, especially since the system makes no guarantees about when the "runtime" download will start/finish.

[samtstern]

Thanks @ParthPadg for the feedback! This was also requested here: https://github.com/firebase/quickstart-android/issues/586

We are tracking this bug internally.

@bjornick what should we do for reports like this that are about SDKs that are not yet part of this repo? Should we file an internal bug and then close them? Leave them open here with a certain label?

[JvmName]

@samtstern Okay! I didn't check the QuickStart repo for duplicates, sorry. Does it matter that firebase/quickstart-android#586 was requesting hooks for custom models, and I'm requesting hooks for models maintained by Firebase?

[samtstern]

@ParthPadg no worries, I would not expect you to check for dupes there.

That's a good clarification, I'll update the internal discussion with that note.

[bjornick]

I think we should file an internal bug for the ML Kit team to use, but leave this issue open until it is fixed. I'm not sure what the right tag to apply here is though.

[samtstern]

@bjornick cool, the internal bug already exists. I added a tracking-internally label to signify that this is not the source of truth for work on this item (since the MLKit team does not do SDK work on GitHub)

[jacobg]

Any update on this issue? Is it possible to bundle a default model (e.g., face) in the app binary, and bypass this whole issue?

[p30arena]

We need an offline model bundled with the app for users with no network access.

[rickdroio]

is there any update?

[astamato]

Any updates? we're getting some errors and edge cases due to models not being ready on time. We could really use some api indicating the state of thee models, even FirebaseVisionBarcodeDetector.isOperational() method in the same fashion Google Mobile Vision has.

[frist008]

Any updates? My users very often get this error. Approximately 10% of users. This is a lot!

[LintonAchmad]

This should be solved with the new FirebaseModelManager API.

https://github.com/firebase/quickstart-android/blob/master/mlkit/app/src/main/java/com/google/firebase/samples/apps/mlkit/java/custommodel/CustomImageClassifier.java#L137

[frist008]

This should be solved with the new FirebaseModelManager API.

https://github.com/firebase/quickstart-android/blob/master/mlkit/app/src/main/java/com/google/firebase/samples/apps/mlkit/java/custommodel/CustomImageClassifier.java#L137

This example raises many questions. I will look at the source code a bit later to understand how it works.

For example, for firebaseModelManager.isModelDownloaded, there is a constant: final String REMOTE_FLOAT_MODEL_NAME = "mobilenet_float_v2_1.0_299" ;:

1. Is it only for my models, or for standard?
2. And how do I know the version for standard models?
3. How do I know what this model is for. In this example, nothing is understood by name? ocr, image or barcode?

Perhaps past questions are superfluous ...

4. For ready-made models, should I use FirebaseAutoMLRemoteModel? I read the builder constructor and don’t understand what SMART_REPLY_MODEL_HASH is. Do I need to send there for example ocr?

[vlafourcade]

Another reason to have an embedded model is because some users download apps from different app stores (e.g. HiMarket, Samsung, Tencent, etc.) and in most cases don't have the Google Play Services installed. So I'd like to reinforce the necessity to have an embedded model (it may be a lightweight version, just to have the feature working properly) to avoid this kind of issues.

[Warpzit]

This is also an issue when the app is downloaded through any MDM system. The current API forces us to use another OCR solution.

[samtstern]

One reasons some downloads may be failing is if your device has an out of date version of Google Play services.

You can use the methods of GoogleApiAvailability to check if you have the right version and prompt users to update: https://developers.google.com/android/reference/com/google/android/gms/common/GoogleApiAvailability#isGooglePlayServicesAvailable(android.content.Context,%20int)