convert TF2 ssd_mobilenet_v2 to tflite

[SajjadAemmi]

Hi,

I am trying to convert a '_ssd_mobilenet_v2_320x320_coco17tpu-8' TF2 model to .tflite.

i use export_tflite_ssd_graph.py for create tflite_graph.pb

But this either fails to create. I am working in Tensorflow 2.3.0 and Python 3.7.3.

[sourangshupal]

Hello, Even I was trying to convert the checkpoint to .tflite.

I converted successfully using "exporter_main_v2.py".

But when converting it to tflite I am facing issues.

Model I am using is _"ssd_resnet50_v1_fpn_640x640_coco17tpu-8"

I am using the export_tflite_ssd_graph.py to creating tflite file

In the script, I have disabled eager execution also.

Then I get the error

File "export_tflite_ssd_graph.py", line 148, in <module>
    tf.app.run(main)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/platform/app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.6/dist-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "export_tflite_ssd_graph1.py", line 144, in main
    FLAGS.max_classes_per_detection, use_regular_nms=FLAGS.use_regular_nms)
  File "/usr/local/lib/python3.6/dist-packages/object_detection-0.1-py3.6.egg/object_detection/export_tflite_ssd_graph_lib.py", line 282, in export_tflite_graph
    exporter.rewrite_nn_resize_op(is_quantized)
  File "/usr/local/lib/python3.6/dist-packages/object_detection-0.1-py3.6.egg/object_detection/exporter.py", line 145, in rewrite_nn_resize_op
    while remove_nn():
  File "/usr/local/lib/python3.6/dist-packages/object_detection-0.1-py3.6.egg/object_detection/exporter.py", line 100, in remove_nn
    input_pattern = graph_matcher.OpTypePattern(
NameError: name 'graph_matcher' is not defined

[ravikyram]

@sajjadaemmi

I have tried in colab with TF nightly (2.4.0-dev20200803) and am not seeing issue.Please, find the gist here.Thanks!

[sourangshupal]

@ravikyram Thanks it worked me..The model size is around 50MB.

Can you help me with how to inference with the tflite on a single image?

Any help with scripts or gist is appreciated

Thanks in advance

[ravikyram]

@sourangshupal

Can you please raise an new issue by filling issue template.Thanks!

[SajjadAemmi]

@ravikyram it worked. but get another error when load model:

import tensorflow as tf
interpreter = tf.lite.Interpreter(model_path)

ValueError: Didn't find op for builtin opcode 'CONV_2D' version '5'
Registration failed.

[AIProCo]

@ravikyram

I think many researchers are suffering from the tflite issues of TF2. It would be very helpful if you give a description of how to generate tflite and do inference with the generated tflite file. I think this description should be included in tf2.md as a guide...

[sourangshupal]

@AIProS Yes, you are correct facing issues in tf2 with tflite. A description will be really helpful.

[ravikyram]

@sajjadaemmi

Can you try with

interpreter = tf.lite.Interpreter(model_path="./model.tflite")
interpreter.allocate_tensors()

Thanks!

[ravikyram]

Please, go through the link,link1 and see if it helps you.Thanks!

[AIProCo]

Dear @ravikyram

Thank you for your quick reply. I read the links before and the link1 is only valid for TF1. Please, if you can inference using a tflite of TF2, give the sample code or gist . It would be very helpful.

[sourangshupal]

@sajjadaemmi

Can you try with

interpreter = tf.lite.Interpreter(model_path="./model.tflite")
interpreter.allocate_tensors()

Thanks!

in TF2 in getting this error

[srjoglekar246]

@sajjadaemmi @sourangshupal

I am working on a script to convert TF2 models to TFLite. Will ping back with updates.

[SajjadAemmi]

@srjoglekar246 Hurry up please

[srjoglekar246]

@sajjadaemmi Out of curiosity, is there a specific reason you prefer the TF2 model over the TF1 versions? The Edge TPU mobiledet will probably give you a comparable mAP, though its trained on COCO'14.

[ValentinBlokhin]

@srjoglekar246 I'm having the same issue, is there any progress on your side?

[srjoglekar246]

There should be a fix coming next week or so. Sorry for the delay, this was a tricky problem due to TF2 APIs and our MLIR converter both being new pieces :-).

[AMArostegui]

Same problem here. Sadly, for now I'm back to TF1 until the scripts are available in the future.

[ichitaka]

@AMArostegui how did you get it to work on TF1 ? Sadly I have the same issue right now with TF2 but a TF1 conversion is also not working on TF Lite for Android.

[srjoglekar246]

@ichitaka Are you using the tflite instructions for ODAPI? There is an intermediate step for exporting a TFLite-friendly version of the detection model.

[AMArostegui]

@AMArostegui how did you get it to work on TF1 ? Sadly I have the same issue right now with TF2 but a TF1 conversion is also not working on TF Lite for Android.

Well, obviously I had to change _finetunning model chekpoints to one of those available in TF1 Zoo and use TF1 legacy scripts to create the model.

Apart from that, I had some difficulties to setup TF1 Python virtualenv, but when everything was done, I had no problem following the instructions in: https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/android

[SajjadAemmi]

@srjoglekar246 when it will be ready? we need it very much

[srjoglekar246]

The code is in internal review, might take a week to land

[celalutku]

The code is in internal review, might take a week to land

I just want to convert my model (transfer learned object detection at TF2) to tflite model and use it in your android demo. Is the new code capable of doing so?

Thank you

[srjoglekar246]

@celalutku Yup!

[shahidammer]

I'm also getting this **_NameError: name 'graph_matcher' is not defined_** when I am trying to export it to tflite (android) model: ssd_resnet50_v1_fpn_640x640_coco17_tpu-8 tt_nightly Python 3.8

[raz-SX]

The code is in internal review, might take a week to land

@srjoglekar246 - any updates?

[terryzhangthu]

Any updates? Met the same issue when converting a "ssd_mobilenet_v2" saved_model to tflite by "export_tflite_ssd_graph.py"

[hahmad2008]

The code is in internal review, might take a week to land

Hello @srjoglekar246 , Any update?

[srjoglekar246]

Our MLIR converter for TFLite has been updated. The detection team is reviewing the script to export the intermediate SavedModel (similar to what export_tflite_ssd_graph does today)

[srjoglekar246]

The script just landed, and it currently only supports the SSD models. We plan to look into other architectures soon...

Could you try conversion and let me know if things work?

[SajjadAemmi]

@srjoglekar246 export_tflite_graph_tf2.py script workes well on ssd_mobilenet_v2_320x320_coco17_tpu-8 but you should change --pipeline_config_path to --pipeline_config_dir in readme

now convert to tflite not worked. how i can use this?

[srjoglekar246]

@sajjadaemmi Thanks for catching that! For conversion, use the converter on command line as:

tflite_convert \
  --saved_model_dir=/tmp/mobilenet_saved_model \
  --output_file=/tmp/mobilenet.tflite

For post-training quantization, use the converter's Python API.

[SajjadAemmi]

it makes a file with 2kb size :-1:

[srjoglekar246]

Can you paste the exporting & conversion commands you ran?

[SajjadAemmi]

i run this command in google colab:

!tflite_convert \
  --saved_model_dir='/content/drive/My Drive/object_detection/inference_graph_tflite_tf2/saved_model'\
  --output_file='/content/drive/My Drive/object_detection/tflite_tf2/model.tflite'

i zepped the saved_model and sent for you for testing

saved_model.zip

[srjoglekar246]

Seems to be working for me: model.tflite.zip

If your TFLite version is old, you might need to specify --experimental_new_converter during conversion.

[mihir-chauhan]

@srjoglekar246 Thank you so much for helping us fix this issue!

When I run the tflite_convert, for some reason, it makes the .tflite file around 500-600 bytes. What would be causing this? My original saved_model is around 7,000 KB. Would the .tflite still work?

[srjoglekar246]

@mihir-chauhan That is unexpected, if you are using our new converter (enabled by --experimental_new_converter=True for older versions of TF). Were you able to generate the SavedModel using the exporting script, around 20-30MB in size?

[mihir-chauhan]

@srjoglekar246 My SavedModel is around 7,000 KB. I am about to try with:

tflite_convert \ --saved_model_dir=C:/tmp/test/saved_model \ --output_file=C:/tmp/mobilenet.tflite \ --experimental_new_converter=True

[srjoglekar246]

The SavedModel should probably be larger. Can you paste your command to export?

[mihir-chauhan]

Sure! Here it is:

python object_detection/export_tflite_graph_tf2.py \ --pipeline_config_path C:/Data/TFOD/models/output_inference_graph/pipeline.config \ --trained_checkpoint_dir C:/Data/TFOD/models/output_inference_graph/checkpoint \ --output_directory C:/tmp/test

[mihir-chauhan]

By the way, my original model from which I use the export_tflite_graph_tf2.py is only around 7,500 KB, but that shouldn't cause the tflite file to be just 500-600 bytes right?

[srjoglekar246]

True. Have you confirmed that the original SavedModel does inference correctly?

[mihir-chauhan]

I believe I did do inference on it and did it correctly.

[srjoglekar246]

I see. In that case, unless there were any error logs from the exporter or converter, the TFLite model should be fine. Its difficult to say what might have gone wrong from the converter without the input SavedModel to inspect :-)

You could probably use Netron to see if the TFLite model has ops you expect, and benchmark it as a sanity test.

[mihir-chauhan]

Thank you, @srjoglekar246. Could you please try converting this model and see if you also result in a ~500 byte tflite model? That would be really great :) saved_model.zip

[srjoglekar246]

@mihir-chauhan Was able to convert your model: model.tflite.zip (11MB).

For the exporting script, you need to combine the checkpoint & ckpt-0 files under a common directory, and use that as trained_checkpoint_dir.

[mihir-chauhan]

Ok thanks, @srjoglekar246! I'll test again and let you know if it works.

[mihir-chauhan]

@srjoglekar246, I am actually running into the same problem. This time, I have ckpt and checkpoint in the same checkpoint folder directory. I have tried it multiple times and it still gives the same 500 byte file. Can you please try converting this to a tflite? I did the exporter script already. I just want to see if the problem is before exporting or after exporting using export_tflite_graph_tf2.py. Thanks again! saved_model_after_exporter_script.zip

[mihir-chauhan]

@jennya, @srjoglekar246 from the Tensorflow Team made a script to do this. That README has the instructions to use it. I am still having some problems with that, however. Link to README