How to run Tensorflow Lite models?

[DevelopDaily]

I cannot run TF Lite models with frugally-deep. For example, this pre-trained model https://storage.googleapis.com/download.tensorflow.org/models/tflite/coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.zip cannot be run.

Is there a workaround to run it?

[Dobiasd]

Yeah, trying to convert this model for frugally-deep:

python3 ./keras_export/convert_model.py coco_ssd_mobilenet_v1_1.0_quant_2018_06_29/detect.tflite model.json

Ends in this error:

OSError: SavedModel file does not exist at: coco_ssd_mobilenet_v1_1.0_quant_2018_06_29/detect.tflite/{saved_model.pbtxt|saved_model.pb}

Frugally-deep only works with model, that can be loaded using tensorflow.keras.models.load_model.

In order to use the model with frugally-deep, you'd need to convert it to a normal Keras model first, but according to this answer on StackOverflow.

The conversion from a TensorFlow SaveModel or tf.keras H5 model to .tflite is an irreversible process.

[DevelopDaily]

I am glad you are aware of the issue. I think your project would be very suitable for small systems like Raspberry Pi. Right now, I have to use Tensorflow Lite to run that model on Raspberry Pi to do the object detection.

I hope you would make Frugally-Deep to run the Tensorflow Lite directly in the future.

[Dobiasd]

Thanks. I was aware of the fact (frugally-deep not being compatible with TensorFlow lite models), but I was not aware that this is an issue.

I thought a TFlite model originates from a "normal" model, so people can use this original model (loadable by tensorflow.keras.models.load_model) with frugally-deep.

So I saw these two paths:

• model.h5 (Keras) ----tflite_convert----> model.tflite ----> use with TensorFlow lite
• model.h5 (Keras) ----frugally-deep/convert_model.py----> model.json ----> use with frugally-deep

In what situation does one need the below path?

• model.tflite ----frugally-deep/convert_model.py----> model.json ----> use with frugally-deep

Do the original models (model.h5) easily get lost, or do some people not publish them and only publish the already-converted model.tflite file?

[DevelopDaily]

Some people only publish the already-converted model.tflite files. This is the problem. Their model.h5 counterparts must be somewhere, but are not made available by their publishers. The frugally-deep is a very cool project. I don't intend to distract you with non-essential issues.

So, I guess this is not a priority issue. On the other hand, if the model.tflite ---> ... ---> model.json path is super easy, it may be worth trying it.

[Dobiasd]

Ah ok. I wonder if they do this on purpose (to keep something secret) or just because they think nobody would need the original model. In case it's the latter, if might be worth if you simply ask if they can publish the original model too.

Another way, since single-shot detection with a mobilenet does not sound too exotic, might be to try to find the original model somewhere else online by searching for ssd mobilenet keras model or similar.

I'll also check if there's a way to convert a TFLite model to Keras (or to fdeep directly), but so far it does not look too good.

[DevelopDaily]

I'd like to rephrase the issue a little. Ultimately, the purpose is to run the pretrained TF models easily. Your h5 approach is straightforward. But, it could be even easier if you could make a script to convert the models published on the model marketplace such as TensorFlow Hub (https://tfhub.dev) to frugally-deep models directly.

Right now, the TensorFlow Hub API allows this:

import tensorflow_hub as hub
hub_model = hub.load('https://tfhub.dev/tensorflow/ssd_mobilenet_v2/fpnlite_640x640/1')

From there, it is still difficult to export that to the model that frugally-deep can run.

It would be nice to do something like this:

frugally_deep_utils.tf_to_fd('https://tfhub.dev/tensorflow/ssd_mobilenet_v2/fpnlite_640x640/1', file_path_json_for_frugally_deep)

[Dobiasd]

Thanks for the explanation. :+1:

Frugally-deep is built entirely around emulating exactly what Keras layers do. While, in the end, Keras layers are implemented in TensorFlow too, it might be quite difficult to "generalize" frugally-deep in a way, that it can work with arbitrary TensorFlow models instead. :grimacing:

But I'll have a look at the problem nonetheless. :mechanic: