Cannot convert to tflite, problem with lambda

mbartosi commented 5 years ago

Hi, I want to convert weights to tflite using tflite_convert --keras_model_file=yolov3-tiny.h5 --output_file=yolov3-tiny.tflite It fails with the following traceback:

Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0513 14:27:21.003883 140717950494528 deprecation.py:506] From /usr/local/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:97: calling Zeros.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor W0513 14:27:21.004970 140717950494528 deprecation.py:506] From /usr/local/lib/python3.7/site-packages/tensorflow/python/ops/init_ops.py:97: calling Ones.init (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor /usr/local/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:820: UserWarning: yolov3_tf2.models is not loaded, but a Lambda layer uses it. It may cause errors. , UserWarning) Traceback (most recent call last): File "/usr/local/bin/tflite_convert", line 11, in sys.exit(main()) File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/tflite_convert.py", line 448, in main app.run(main=run_main, argv=sys.argv[:1]) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/platform/app.py", line 40, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/usr/local/lib/python3.7/site-packages/absl/app.py", line 300, in run _run_main(main, args) File "/usr/local/lib/python3.7/site-packages/absl/app.py", line 251, in _run_main sys.exit(main(argv)) File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/tflite_convert.py", line 444, in run_main _convert_model(tflite_flags) File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/tflite_convert.py", line 123, in _convert_model converter = _get_toco_converter(flags) File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/tflite_convert.py", line 110, in _get_toco_converter return converter_fn(converter_kwargs) File "/usr/local/lib/python3.7/site-packages/tensorflow/lite/python/lite.py", line 627, in from_keras_model_file keras_model = _keras.models.load_model(model_file) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/saving/hdf5_format.py", line 215, in load_model custom_objects=custom_objects) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/saving/model_config.py", line 55, in model_from_config return deserialize(config, custom_objects=custom_objects) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/layers/serialization.py", line 95, in deserialize printable_module_name='layer') File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/utils/generic_utils.py", line 192, in deserialize_keras_object list(custom_objects.items()))) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1231, in from_config process_layer(layer_data) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1215, in process_layer layer = deserialize_layer(layer_data, custom_objects=custom_objects) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/layers/serialization.py", line 95, in deserialize printable_module_name='layer') File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/utils/generic_utils.py", line 192, in deserialize_keras_object list(custom_objects.items()))) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1241, in from_config process_node(layer, node_data) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/network.py", line 1197, in process_node layer(flat_input_tensors[0], kwargs) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 612, in call outputs = self.call(inputs, *args, kwargs) File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py", line 768, in call return self.function(inputs, arguments) File "/home/mba/GitHub/yolov3-tf2/yolov3_tf2/models.py", line 139, in x = Lambda(lambda x: import tensorflow as tf; tf.reshape(x, (-1, tf.shape(x)[1], tf.shape(x)[2], anchors, classes + 5)))(x) NameError: name 'tf' is not defined

How could I solve it?

zzh8829 commented 5 years ago

I tried running tflite_convert in tensorflow 2.0 it says tflite_convert is currently unsupported in 2.0

Following the instruction on TF2.0 API export to tfserving saved_model first

python export_tfserving.py --output serving/yolov3/1/

run these in python

import tensorflow as tf
model = tf.saved_model.load("serving/yolov3/1")
converter = tf.lite.TFLiteConverter.from_concrete_function(model.signatures['serving_default'])
tflite_model = converter.convert() 
open("converted_model.tflite", "wb").write(tflite_model)

This should work but i'm getting weird errors

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/lite/python/lite.py", line 261, in convert
    shape_list = tensor.get_shape().as_list()
  File "/usr/local/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/framework/tensor_shape.py", line 1128, in as_list
    raise ValueError("as_list() is not defined on an unknown TensorShape.")
ValueError: as_list() is not defined on an unknown TensorShape.

¯_(ツ)_/¯

Sorry I can't really help right now, i guess we will have to wait for new updates from tflite 2.0.

gwestner94 commented 5 years ago

Is there any update for this? I would love to try using the model in tflite.

zzh8829 commented 5 years ago

I followed the new instruction with model support https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/lite/TFLiteConverter

model = YoloV3()
model.load_weights('weights.tf')
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

But unfortunately i got this error

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted

You can try it too and see if it works

Lap1n commented 5 years ago

Did you guys managed to convert it to TFlite? I'm getting this error :

2019-08-28 11:02:00.428562: F tensorflow/contrib/lite/toco/tflite/export.cc:374] Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.contrib.lite.TocoConverter(). Here is a list of operators for which  you will need custom implementations: AddV2, FusedBatchNormV3, LeakyRelu, ResizeNearestNeighbor.

ghost commented 5 years ago

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

ghost commented 5 years ago

Did you guys managed to convert it to TFlite? I'm getting this error :

2019-08-28 11:02:00.428562: F tensorflow/contrib/lite/toco/tflite/export.cc:374] Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.contrib.lite.TocoConverter(). Here is a list of operators for which  you will need custom implementations: AddV2, FusedBatchNormV3, LeakyRelu, ResizeNearestNeighbor.

Hey, Tensorflow 2.0 RC1 is released and all my error messages have now disappeared. Can anyone confirm that?

1165048017 commented 5 years ago

For "NameError: name 'tf' is not defined" you can add the 'custom_objects' parameter in load_model function: tf.keras.models.load_model('xxx.h5',custom_objects={"tf":tf})

I also met the problem 'keras is not defined' while load model, the same method to solve it: tf.keras.models.load_model('xxx.h5',custom_objects={"keras":tf.keras})

peace195 commented 5 years ago

Here is my solution: https://github.com/peace195/tensorflow-lite-yolo-v3

The .pb should be in right format (SavedModel).

Please try it. I would appreciate if you give me a star for this project 👍

JJ306 commented 5 years ago

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

Hi,

You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

ghost commented 5 years ago

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)
For me, this works, but I am still trying to implement it to android. Maybe one of you can help?
Hi,

You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

Which Version of Tensorflow do you use?

JJ306 commented 5 years ago

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)
For me, this works, but I am still trying to implement it to android. Maybe one of you can help?
Hi, You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?
Which Version o
yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)
For me, this works, but I am still trying to implement it to android. Maybe one of you can help?
Hi, You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?
Which Version of Tensorflow do you use?

I am using TF2.0 & Anaconda on windows 10.

dao-kun commented 4 years ago

those code is also works for me , but the the tfliite output is totally different, did anyone of you meet this before? `yolo_size = 416 yolo_weights = './checkpoints/yolov3_train_116.tf' tflite_model_path = './weights/yolo.tflite' NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES) yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo) tflite_model = converter.convert() open(tflite_model_path, 'wb').write(tflite_model)`

my tflite input : [{'name': 'input_1', 'index': 3, 'shape': array([ 1, 416, 416, 3], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}]

output :

[{'name': 'Identity', 'index': 0, 'shape': array([ 1, 13, 13, 3, 85], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}, {'name': 'Identity_1', 'index': 1, 'shape': array([ 1, 26, 26, 3, 85], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}, {'name': 'Identity_2', 'index': 2, 'shape': array([ 1, 52, 52, 3, 85], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}]

zzh8829 commented 4 years ago

Looks like TFLite converter has problem with CombinedNonMaxSupression operator, we will have to wait for official support for now

zzh8829 commented 4 years ago

https://github.com/tensorflow/tensorflow/issues/34693