Open Daveorr opened 2 years ago
I will consider implementation.
they have a huge impact on performance when running on older devices.
BTW, what devices does "older devices" refer to?
Also, how did you get around the onnx-tf conversion error? It cannot be converted successfully with onnx-tf.
$ onnx-tf convert -i ffnet_opset12_simplified.onnx -o ffnet_opset12_simplified
/usr/local/lib/python3.8/dist-packages/tensorflow_addons/utils/ensure_tf_install.py:53: UserWarning: Tensorflow Addons supports using Python ops for all Tensorflow versions above or equal to 2.3.0 and strictly below 2.5.0 (nightly versions are not supported).
The versions of TensorFlow you are currently using is 2.9.0 and is not supported.
Some things might work, some things might not.
If you were to encounter a bug, do not file an issue.
If you want to make sure you're using a tested and supported configuration, either change the TensorFlow version or the TensorFlow Addons's version.
You can find the compatibility matrix in TensorFlow Addon's readme:
https://github.com/tensorflow/addons
warnings.warn(
2022-08-04 00:17:22,087 - onnx-tf - INFO - Start converting onnx pb to tf saved model
Traceback (most recent call last):
File "/usr/local/bin/onnx-tf", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/onnx_tf/cli.py", line 20, in main
return onnx_tf.converter.main(args[1:])
File "/usr/local/lib/python3.8/dist-packages/onnx_tf/converter.py", line 21, in main
convert(**{k: v for k, v in vars(args).items() if v is not None})
File "/usr/local/lib/python3.8/dist-packages/onnx_tf/converter.py", line 147, in convert
tf_rep.export_graph(outdir)
File "/usr/local/lib/python3.8/dist-packages/onnx_tf/backend_rep.py", line 143, in export_graph
signatures=self.tf_module.__call__.get_concrete_function(
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 1239, in get_concrete_function
concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 1219, in _get_concrete_function_garbage_collected
self._initialize(args, kwargs, add_initializers_to=initializers)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 785, in _initialize
self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 2480, in _get_concrete_function_internal_garbage_collected
graph_function, _ = self._maybe_define_function(args, kwargs)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 2711, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 2627, in _create_graph_function
func_graph_module.func_graph_from_py_func(
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py", line 1141, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 677, in wrapped_fn
out = weak_wrapped_fn().__wrapped__(*args, **kwds)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 3251, in bound_method_wrapper
return wrapped_fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py", line 1127, in autograph_handler
raise e.ag_error_metadata.to_exception(e)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py", line 1116, in autograph_handler
return autograph.converted_call(
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call
result = converted_f(*effective_args, **kwargs)
File "/tmp/__autograph_generated_file85j3buf8.py", line 30, in tf____call__
ag__.for_stmt(ag__.ld(self).graph_def.node, None, loop_body, get_state, set_state, (), {'iterate_names': 'node'})
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 449, in for_stmt
_py_for_stmt(iter_, extra_test, body, None, None)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 498, in _py_for_stmt
body(target)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 464, in protected_body
original_body(protected_iter)
File "/tmp/__autograph_generated_file85j3buf8.py", line 23, in loop_body
output_ops = ag__.converted_call(ag__.ld(self).backend._onnx_node_to_tensorflow_op, (ag__.ld(onnx_node), ag__.ld(tensor_dict), ag__.ld(self).handlers), dict(opset=ag__.ld(self).opset, strict=ag__.ld(self).strict), fscope)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call
result = converted_f(*effective_args, **kwargs)
File "/tmp/__autograph_generated_file21606sap.py", line 50, in tf___onnx_node_to_tensorflow_op
ag__.if_stmt(ag__.ld(handlers), if_body_1, else_body_1, get_state_1, set_state_1, ('do_return', 'retval_'), 2)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1341, in if_stmt
_py_if_stmt(cond, body, orelse)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1394, in _py_if_stmt
return body() if cond else orelse()
File "/tmp/__autograph_generated_file21606sap.py", line 44, in if_body_1
ag__.if_stmt(ag__.ld(handler), if_body, else_body, get_state, set_state, ('do_return', 'retval_'), 2)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1341, in if_stmt
_py_if_stmt(cond, body, orelse)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1394, in _py_if_stmt
return body() if cond else orelse()
File "/tmp/__autograph_generated_file21606sap.py", line 36, in if_body
retval_ = ag__.converted_call(ag__.ld(handler).handle, (ag__.ld(node),), dict(tensor_dict=ag__.ld(tensor_dict), strict=ag__.ld(strict)), fscope)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call
result = converted_f(*effective_args, **kwargs)
File "/tmp/__autograph_generated_filetk6dixjb.py", line 34, in tf__handle
ag__.if_stmt(ag__.ld(ver_handle), if_body, else_body, get_state, set_state, ('do_return', 'retval_'), 2)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1341, in if_stmt
_py_if_stmt(cond, body, orelse)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1394, in _py_if_stmt
return body() if cond else orelse()
File "/tmp/__autograph_generated_filetk6dixjb.py", line 26, in if_body
retval_ = ag__.converted_call(ag__.ld(ver_handle), (ag__.ld(node),), dict(**ag__.ld(kwargs)), fscope)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call
result = converted_f(*effective_args, **kwargs)
File "/tmp/__autograph_generated_fileistomi9h.py", line 12, in tf__version
retval_ = ag__.converted_call(ag__.ld(cls)._common, (ag__.ld(node),), dict(**ag__.ld(kwargs)), fscope)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call
result = converted_f(*effective_args, **kwargs)
File "/tmp/__autograph_generated_file0lt6nnqf.py", line 123, in tf___common
cond = ag__.converted_call(ag__.ld(tf).cond, (ag__.converted_call(ag__.ld(check_positive), (ag__.ld(paddings),), None, fscope), ag__.autograph_artifact((lambda : ag__.converted_call(ag__.ld(process_pos_pads), (ag__.ld(x), ag__.ld(paddings), ag__.ld(constant_values)), None, fscope))), ag__.autograph_artifact((lambda : ag__.converted_call(ag__.ld(process_neg_pads), (ag__.ld(x), ag__.ld(paddings), ag__.ld(constant_values)), None, fscope)))), None, fscope)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 377, in converted_call
return _call_unconverted(f, args, kwargs, options)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 459, in _call_unconverted
return f(*args)
File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/tmp/__autograph_generated_file0lt6nnqf.py", line 123, in <lambda>
cond = ag__.converted_call(ag__.ld(tf).cond, (ag__.converted_call(ag__.ld(check_positive), (ag__.ld(paddings),), None, fscope), ag__.autograph_artifact((lambda : ag__.converted_call(ag__.ld(process_pos_pads), (ag__.ld(x), ag__.ld(paddings), ag__.ld(constant_values)), None, fscope))), ag__.autograph_artifact((lambda : ag__.converted_call(ag__.ld(process_neg_pads), (ag__.ld(x), ag__.ld(paddings), ag__.ld(constant_values)), None, fscope)))), None, fscope)
File "/tmp/__autograph_generated_file0lt6nnqf.py", line 40, in process_neg_pads
result = ag__.converted_call(ag__.ld(tf).slice, (ag__.ld(x), ag__.ld(begins), ag__.ld(sizes)), None, fscope_2)
ValueError: in user code:
File "/usr/local/lib/python3.8/dist-packages/onnx_tf/backend_tf_module.py", line 99, in __call__ *
output_ops = self.backend._onnx_node_to_tensorflow_op(onnx_node,
File "/usr/local/lib/python3.8/dist-packages/onnx_tf/backend.py", line 347, in _onnx_node_to_tensorflow_op *
return handler.handle(node, tensor_dict=tensor_dict, strict=strict)
File "/usr/local/lib/python3.8/dist-packages/onnx_tf/handlers/handler.py", line 59, in handle *
return ver_handle(node, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/onnx_tf/handlers/backend/pad.py", line 91, in version_11 *
return cls._common(node, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/onnx_tf/handlers/backend/pad.py", line 35, in process_neg_pads *
result=tf.slice(x, begins, sizes)
ValueError: Invalid value in tensor used for shape: -4
Also, how did you get around the onnx-tf conversion error? It cannot be converted successfully with onnx-tf.
$ onnx-tf convert -i ffnet_opset12_simplified.onnx -o ffnet_opset12_simplified /usr/local/lib/python3.8/dist-packages/tensorflow_addons/utils/ensure_tf_install.py:53: UserWarning: Tensorflow Addons supports using Python ops for all Tensorflow versions above or equal to 2.3.0 and strictly below 2.5.0 (nightly versions are not supported). The versions of TensorFlow you are currently using is 2.9.0 and is not supported. Some things might work, some things might not. If you were to encounter a bug, do not file an issue. If you want to make sure you're using a tested and supported configuration, either change the TensorFlow version or the TensorFlow Addons's version. You can find the compatibility matrix in TensorFlow Addon's readme: https://github.com/tensorflow/addons warnings.warn( 2022-08-04 00:17:22,087 - onnx-tf - INFO - Start converting onnx pb to tf saved model Traceback (most recent call last): File "/usr/local/bin/onnx-tf", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.8/dist-packages/onnx_tf/cli.py", line 20, in main return onnx_tf.converter.main(args[1:]) File "/usr/local/lib/python3.8/dist-packages/onnx_tf/converter.py", line 21, in main convert(**{k: v for k, v in vars(args).items() if v is not None}) File "/usr/local/lib/python3.8/dist-packages/onnx_tf/converter.py", line 147, in convert tf_rep.export_graph(outdir) File "/usr/local/lib/python3.8/dist-packages/onnx_tf/backend_rep.py", line 143, in export_graph signatures=self.tf_module.__call__.get_concrete_function( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 1239, in get_concrete_function concrete = self._get_concrete_function_garbage_collected(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 1219, in _get_concrete_function_garbage_collected self._initialize(args, kwargs, add_initializers_to=initializers) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 785, in _initialize self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 2480, in _get_concrete_function_internal_garbage_collected graph_function, _ = self._maybe_define_function(args, kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 2711, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 2627, in _create_graph_function func_graph_module.func_graph_from_py_func( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py", line 1141, in func_graph_from_py_func func_outputs = python_func(*func_args, **func_kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/def_function.py", line 677, in wrapped_fn out = weak_wrapped_fn().__wrapped__(*args, **kwds) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/function.py", line 3251, in bound_method_wrapper return wrapped_fn(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py", line 1127, in autograph_handler raise e.ag_error_metadata.to_exception(e) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/func_graph.py", line 1116, in autograph_handler return autograph.converted_call( File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call result = converted_f(*effective_args, **kwargs) File "/tmp/__autograph_generated_file85j3buf8.py", line 30, in tf____call__ ag__.for_stmt(ag__.ld(self).graph_def.node, None, loop_body, get_state, set_state, (), {'iterate_names': 'node'}) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 449, in for_stmt _py_for_stmt(iter_, extra_test, body, None, None) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 498, in _py_for_stmt body(target) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 464, in protected_body original_body(protected_iter) File "/tmp/__autograph_generated_file85j3buf8.py", line 23, in loop_body output_ops = ag__.converted_call(ag__.ld(self).backend._onnx_node_to_tensorflow_op, (ag__.ld(onnx_node), ag__.ld(tensor_dict), ag__.ld(self).handlers), dict(opset=ag__.ld(self).opset, strict=ag__.ld(self).strict), fscope) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call result = converted_f(*effective_args, **kwargs) File "/tmp/__autograph_generated_file21606sap.py", line 50, in tf___onnx_node_to_tensorflow_op ag__.if_stmt(ag__.ld(handlers), if_body_1, else_body_1, get_state_1, set_state_1, ('do_return', 'retval_'), 2) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1341, in if_stmt _py_if_stmt(cond, body, orelse) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1394, in _py_if_stmt return body() if cond else orelse() File "/tmp/__autograph_generated_file21606sap.py", line 44, in if_body_1 ag__.if_stmt(ag__.ld(handler), if_body, else_body, get_state, set_state, ('do_return', 'retval_'), 2) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1341, in if_stmt _py_if_stmt(cond, body, orelse) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1394, in _py_if_stmt return body() if cond else orelse() File "/tmp/__autograph_generated_file21606sap.py", line 36, in if_body retval_ = ag__.converted_call(ag__.ld(handler).handle, (ag__.ld(node),), dict(tensor_dict=ag__.ld(tensor_dict), strict=ag__.ld(strict)), fscope) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call result = converted_f(*effective_args, **kwargs) File "/tmp/__autograph_generated_filetk6dixjb.py", line 34, in tf__handle ag__.if_stmt(ag__.ld(ver_handle), if_body, else_body, get_state, set_state, ('do_return', 'retval_'), 2) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1341, in if_stmt _py_if_stmt(cond, body, orelse) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/operators/control_flow.py", line 1394, in _py_if_stmt return body() if cond else orelse() File "/tmp/__autograph_generated_filetk6dixjb.py", line 26, in if_body retval_ = ag__.converted_call(ag__.ld(ver_handle), (ag__.ld(node),), dict(**ag__.ld(kwargs)), fscope) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call result = converted_f(*effective_args, **kwargs) File "/tmp/__autograph_generated_fileistomi9h.py", line 12, in tf__version retval_ = ag__.converted_call(ag__.ld(cls)._common, (ag__.ld(node),), dict(**ag__.ld(kwargs)), fscope) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call result = converted_f(*effective_args, **kwargs) File "/tmp/__autograph_generated_file0lt6nnqf.py", line 123, in tf___common cond = ag__.converted_call(ag__.ld(tf).cond, (ag__.converted_call(ag__.ld(check_positive), (ag__.ld(paddings),), None, fscope), ag__.autograph_artifact((lambda : ag__.converted_call(ag__.ld(process_pos_pads), (ag__.ld(x), ag__.ld(paddings), ag__.ld(constant_values)), None, fscope))), ag__.autograph_artifact((lambda : ag__.converted_call(ag__.ld(process_neg_pads), (ag__.ld(x), ag__.ld(paddings), ag__.ld(constant_values)), None, fscope)))), None, fscope) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 377, in converted_call return _call_unconverted(f, args, kwargs, options) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/autograph/impl/api.py", line 459, in _call_unconverted return f(*args) File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/tmp/__autograph_generated_file0lt6nnqf.py", line 123, in <lambda> cond = ag__.converted_call(ag__.ld(tf).cond, (ag__.converted_call(ag__.ld(check_positive), (ag__.ld(paddings),), None, fscope), ag__.autograph_artifact((lambda : ag__.converted_call(ag__.ld(process_pos_pads), (ag__.ld(x), ag__.ld(paddings), ag__.ld(constant_values)), None, fscope))), ag__.autograph_artifact((lambda : ag__.converted_call(ag__.ld(process_neg_pads), (ag__.ld(x), ag__.ld(paddings), ag__.ld(constant_values)), None, fscope)))), None, fscope) File "/tmp/__autograph_generated_file0lt6nnqf.py", line 40, in process_neg_pads result = ag__.converted_call(ag__.ld(tf).slice, (ag__.ld(x), ag__.ld(begins), ag__.ld(sizes)), None, fscope_2) ValueError: in user code: File "/usr/local/lib/python3.8/dist-packages/onnx_tf/backend_tf_module.py", line 99, in __call__ * output_ops = self.backend._onnx_node_to_tensorflow_op(onnx_node, File "/usr/local/lib/python3.8/dist-packages/onnx_tf/backend.py", line 347, in _onnx_node_to_tensorflow_op * return handler.handle(node, tensor_dict=tensor_dict, strict=strict) File "/usr/local/lib/python3.8/dist-packages/onnx_tf/handlers/handler.py", line 59, in handle * return ver_handle(node, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx_tf/handlers/backend/pad.py", line 91, in version_11 * return cls._common(node, **kwargs) File "/usr/local/lib/python3.8/dist-packages/onnx_tf/handlers/backend/pad.py", line 35, in process_neg_pads * result=tf.slice(x, begins, sizes) ValueError: Invalid value in tensor used for shape: -4
Yes, I had to modify some files from the original library in order to avoid this error, I've uploaded the modified library in the link above but If you don't wanna go through this hack I've also uploaded the obtained TF/TFlite models in the models
folder always at the same Gdrive link
I will consider implementation.
they have a huge impact on performance when running on older devices.
BTW, what devices does "older devices" refer to?
I've tested it on old android devices such as MI A2 lite
Issue Type
Feature Request
OS
Ubuntu
OS architecture
x86_64
Programming Language
Python
Framework
OpenVINO, PyTorch, ONNX, TensorFlow, TensorFlowLite, Other
Download URL for ONNX / OpenVINO IR
https://drive.google.com/drive/folders/1XFqjFn4OnyLBszWnVkA4uXNsD2RyAXEK?usp=sharing
Convert Script
see provided Gdrive link
Description
Hello!
I'm trying to convert a model from openVINO-IR to Tensorflow(lite) but during conversion I stumble upon the following message
The Loop layer is not yet implemented.
I already managed to convert this model from ONNX to Tensorflow using
onnx2tensorflow
tool (see here) but I would like to use your tool in order to get rid of all those garbageTranspose
layers that occurs when converting fromNCHW
toNHWC
since they have a huge impact on performance when running on older devices.If you are not planning to add this feature anytime soon can you kindly suggest any workaround / improvement that can be made in my case? (model structure, conversion code and anything needed should be in the link provided)
Thanks in advance, happy to share more info if needed!
Relevant Log Output
Source code for simple inference testing code
see provided Gdrive link