search

tensorflow / models

Models and examples built with TensorFlow
Other
76.94k stars 45.8k forks source link

Converting TF 2 Object Detection Model to TFLite #9031

Closed Snixells closed 4 years ago

Snixells commented 4 years ago

Prerequisites

Please answer the following questions for yourself before submitting an issue.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md

2. Describe the bug

I am trying to Convert the SSD ResNet50 V1 FPN 640x640 (RetinaNet50) of the TF 2 Object Detection Zoo to TFLite. My Code is the following: 

converter = tf.lite.TFLiteConverter.from_saved_model('ssd_resnet50_v1_fpn_640x640_coco17_tpu-8/saved_model/',signature_keys=['serving_default'])
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.experimental_new_converter = True
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]

tflite_model = converter.convert()

with tf.io.gfile.GFile('model.tflite', 'wb') as f:
  f.write(tflite_model)

The Convertion is running without any Errors, but when I try to use the TFLite Model with the following Code

interpreter = tf.lite.Interpreter(model_path="./model.tflite")
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample((1,640,640,3)), dtype=np.uint8)
interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)

I am getting this Error: 

ValueError                                Traceback (most recent call last)
<ipython-input-17-44fd9cb644ae> in <module>
     20 rdm_img = np.array(np.random.random_sample((1,640,640,3)), dtype=np.uint8)
     21 # input_data = np.array(np.random.random_sample(input_shape), dtype=np.uint8)
---> 22 interpreter.set_tensor(input_details[0]['index'], rdm_img)
     23 
     24 interpreter.invoke()

/usr/lib/python3.6/site-packages/tensorflow/lite/python/interpreter.py in set_tensor(self, tensor_index, value)
    406       ValueError: If the interpreter could not set the tensor.
    407     """
--> 408     self._interpreter.SetTensor(tensor_index, value)
    409 
    410   def resize_tensor_input(self, input_index, tensor_size, strict=False):

ValueError: Cannot set tensor: Dimension mismatch. Got 640 but expected 1 for dimension 1 of input 0.

It seems that there are some Problems with the Dimensions of the converted Model. Thanks for your help!

6. System information

ravikyram commented 4 years ago

I tried in colab with TF nightly verison(2.4.0-dev20200803) and was able to reproduce the issue.Please, find the gist here.Thanks!

AIProCo commented 4 years ago

I am facing the same problem...

MeghnaNatraj commented 4 years ago

@Snixells Can you try this?

!pip install tf-nightly
import tensorflow as tf

## TFLite Conversion
model = tf.saved_model.load("saved_model")
concrete_func = model.signatures[tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([1, 640, 640, 3])
tf.saved_model.save(model, "saved_model_updated", signatures={"serving_default":concrete_func})
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir='saved_model_updated', signature_keys=['serving_default'])

converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()

## TFLite Interpreter to check input shape
interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# Test the model on random input data.
input_shape = input_details[0]['shape']
print(input_shape)
ravikyram commented 4 years ago

@Snixells

I have tried in colab with TF nightly version as per @MeghnaNatraj suggestion and i am not seeing any issue.Please, find the gist here. Please, verify once and close the issue.Thanks!

shahidammer commented 4 years ago

@MeghnaNatraj I used your give script to export the model to a tflite but I am getting this

`2020-08-28 11:20:09.890462: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14636 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:04.0, compute capability: 7.0) 2020-08-28 11:20:10.114986: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:816] Optimization results for grappler item: graph_to_optimize 2020-08-28 11:20:10.115049: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:818] function_optimizer: Graph size after: 4187 nodes (3829), 6745 edges (6380), time = 131.197ms. 2020-08-28 11:20:10.115057: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:818] function_optimizer: function_optimizer did nothing. time = 2.969ms. 2020-08-28 11:20:15.209765: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:313] Ignored output_format. 2020-08-28 11:20:15.209826: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:316] Ignored drop_control_dependency. loc("Func/StatefulPartitionedCall/input/_0"): error: requires all operands and results to have compatible element types Traceback (most recent call last): File "/home/dev/tf_nightly_env/lib/python3.8/site-packages/tensorflow/lite/python/convert.py", line 196, in toco_convert_protos model_str = wrap_toco.wrapped_toco_convert(model_flags_str, File "/home/dev/tf_nightly_env/lib/python3.8/site-packages/tensorflow/lite/python/wrap_toco.py", line 32, in wrapped_toco_convert return _pywrap_toco_api.TocoConvert( Exception: :0: error: loc("Func/StatefulPartitionedCall/input/_0"): requires all operands and results to have compatible element types

:0: note: loc("Func/StatefulPartitionedCall/input/_0"): see current operation: %1 = "tf.Identity"(%arg0) {_class = ["loc:@Func/StatefulPartitionedCall/StatefulPartitionedCall/input/_359"], device = ""} : (tensor<1x640x640x3x!tf.quint8>) -> tensor<1x640x640x3xui8> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "tflite_convertor.py", line 13, in tflite_model = converter.convert() File "/home/dev/tf_nightly_env/lib/python3.8/site-packages/tensorflow/lite/python/lite.py", line 1076, in convert return super(TFLiteConverterV2, self).convert() File "/home/dev/tf_nightly_env/lib/python3.8/site-packages/tensorflow/lite/python/lite.py", line 899, in convert return super(TFLiteFrozenGraphConverterV2, File "/home/dev/tf_nightly_env/lib/python3.8/site-packages/tensorflow/lite/python/lite.py", line 629, in convert result = _toco_convert_impl( File "/home/dev/tf_nightly_env/lib/python3.8/site-packages/tensorflow/lite/python/convert.py", line 569, in toco_convert_impl data = toco_convert_protos( File "/home/dev/tf_nightly_env/lib/python3.8/site-packages/tensorflow/lite/python/convert.py", line 202, in toco_convert_protos raise ConverterError(str(e)) tensorflow.lite.python.convert.ConverterError: :0: error: loc("Func/StatefulPartitionedCall/input/_0"): requires all operands and results to have compatible element types :0: note: loc("Func/StatefulPartitionedCall/input/_0"): see current operation: %1 = "tf.Identity"(%arg0) {_class = ["loc:@Func/StatefulPartitionedCall/StatefulPartitionedCall/input/_359"], device = ""} : (tensor<1x640x640x3x!tf.quint8>) -> tensor<1x640x640x3xui8> ` model: **ssd_resnet50_v1_fpn_640x640_coco17_tpu-8** **tt_nightly** Python 3.8