Model details used for object detection are missing

[mrdc]

Hi! Can you, please, tell me which model is used for object detection? I've built the test app and it uses object_labeler.tflite from assets/ml for detection, but it's not clear which model is used: extracted metadata gave no insights, probability-labels-en.txt and probability-labels.txt also gave no info and these files are present in tests in java in 4 repos only across whole Github. Also checked Google ML docs, checked the repo code for insights - nothing. No url for this model in example source code . Other models have urls:

 'birds': 'lite-model_aiy_vision_classifier_birds_V1_3.tflite',
    // https://tfhub.dev/google/lite-model/aiy/vision/classifier/birds_V1/3

    'food': 'lite-model_aiy_vision_classifier_food_V1_1.tflite',
    // https://tfhub.dev/google/lite-model/aiy/vision/classifier/food_V1/1

    'plants': 'lite-model_aiy_vision_classifier_plants_V1_3.tflite',
    // https://tfhub.dev/google/lite-model/aiy/vision/classifier/plants_V1/3

    'mushrooms': 'lite-model_models_mushroom-identification_v1_1.tflite',
    // https://tfhub.dev/bohemian-visual-recognition-alliance/lite-model/models/mushroom-identification_v1/1

    'landmarks':
        'lite-model_on_device_vision_classifier_landmarks_classifier_north_america_V1_1.tflite',
    // https://tfhub.dev/google/lite-model/on_device_vision/classifier/landmarks_classifier_north_america_V1/1

Expected behavior Models, used for detection, have details: origin, url, etc.

Platform (please complete the following information):

• OS: iOS
• Device: iPhone
• OS: iOS 16.7.7
• Flutter/Dart Version 3.19.6/3.3.4
  • Plugin version 0.16.3

Thanks!

[fbernaly]

Did you run and debug the example app?

_option defines which model to use:

https://github.com/flutter-ml/google_ml_kit_flutter/blob/develop/packages/example/lib/vision_detector_views/object_detector_view.dart#L22-L24

_option is used in _initializeDetector()

https://github.com/flutter-ml/google_ml_kit_flutter/blob/develop/packages/example/lib/vision_detector_views/object_detector_view.dart#L130-L173

When the detector is initialized then this callback is called with the InputImage to process:

https://github.com/flutter-ml/google_ml_kit_flutter/blob/develop/packages/example/lib/vision_detector_views/object_detector_view.dart#L176-L210

The detector will return a list with the detected objects as explained here:

https://github.com/flutter-ml/google_ml_kit_flutter/tree/develop/packages/google_mlkit_object_detection

For more details on Custom models go here:

• https://developers.google.com/ml-kit/custom-models
• https://github.com/flutter-ml/google_ml_kit_flutter/tree/develop/packages/google_mlkit_object_detection#local-custom-model

[mrdc]

Did you run and debug the example app?

Hi, yes, it works well. The question is that it's not clear, which model object_labeler.tflite is. I mean tech details like which dataset was used for training, etc. Extracted probability labels give me some basic understanding about model capabilities, but I can't find a source of this model: where was it downloaded from etc.

[fbernaly]

Sorry, I do not have that info and I dont remember where I grabbed that model from, but anyways, those are the models I found for the example, if you want to use a custom model you need to build it or find one that suits your need. I dont recommend using the ones in the example since I dont remeber their source or metadata.