dino-illumix commented 1 year ago

Checklist

[X] I have searched related issues but cannot get the expected help.
[X] 2. I have read the FAQ documentation but cannot get the expected help.
[X] 3. The bug has not been fixed in the latest version.

Describe the bug

Conversion of Mobilenet_v3 models created with mmsegmentation converted using mmdeploy pytorch to ONNX converter appear to have been coverted correctly but then fail during conversion to tensorflow:

import onnx from onnx_tf.backend import prepare import tensorflow as tf

Load the ONNX model

onnx_model = onnx.load(onnx_model_path)

Export TF model:

tf_rep = prepare(onnx_model) tf_rep.export_graph(tf_model_path)

produces the following error: For Mobilenet_v2 models it works fine.

(mmdeploy) D:\Projects\mmsegmentation>python illumix_utils/convert_onnx_to_tflite.py C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow_addons\utils\tfa_eol_msg.py:23: UserWarning:

TensorFlow Addons (TFA) has ended development and introduction of new features. TFA has entered a minimal maintenance and release mode until a planned end of life in May 2024. Please modify downstream libraries to take dependencies from other repositories in our TensorFlow community (e.g. Keras, Keras-CV, and Keras-NLP).

For more information see: https://github.com/tensorflow/addons/issues/2807

warnings.warn( 2023-09-08 17:08:01.657263: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. Traceback (most recent call last): File "D:\Projects\mmsegmentation\illumix_utils\convert_onnx_to_tflite.py", line 43, in tf_rep.export_graph(tf_model_path) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\onnx_tf\backend_rep.py", line 143, in export_graph signatures=self.tf_module.call.get_concrete_function( File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\polymorphic_function.py", line 1189, in get_concrete_function concrete = self._get_concrete_function_garbage_collected(*args, kwargs) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\polymorphic_function.py", line 1169, in _get_concrete_function_garbage_collected self._initialize(args, kwargs, add_initializers_to=initializers) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\polymorphic_function.py", line 694, in _initialize self._variable_creation_fn # pylint: disable=protected-access File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\tracing_compiler.py", line 176, in _get_concrete_function_internal_garbage_collected concretefunction, = self._maybe_define_concrete_function(args, kwargs) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\tracing_compiler.py", line 171, in _maybe_define_concrete_function return self._maybe_define_function(args, kwargs) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\tracing_compiler.py", line 398, in _maybe_define_function concrete_function = self._create_concrete_function( File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\tracing_compiler.py", line 305, in _create_concrete_function func_graph_module.func_graph_from_py_func( File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\framework\func_graph.py", line 1055, in func_graph_from_py_func func_outputs = python_func(*func_args, *func_kwargs) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\polymorphic_function.py", line 597, in wrapped_fn out = weak_wrapped_fn().wrapped(args, kwds) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\tracing_compiler.py", line 453, in bound_method_wrapper return wrapped_fn(*args, kwargs) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\autograph_util.py", line 52, in autograph_handler raise e.ag_error_metadata.to_exception(e) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\eager\polymorphic_function\autograph_util.py", line 41, in autograph_handler return api.converted_call( File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 439, in converted_call result = converted_f(*effective_args, *kwargs) File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_file9n_szo61.py", line 30, in tf__call ag.for_stmt(ag__.ld(self).graph_def.node, None, loop_body, get_state, set_state, (), {'iterate_names': 'node'}) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 454, in for_stmt forfn(iter, extra_test, body, get_state, set_state, symbol_names, opts) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 505, in _py_for_stmt body(target) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 471, in protected_body original_body(protected_iter) File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_file9n_szo61.py", line 23, in loop_body output_ops = ag.converted_call(ag.ld(self).backend._onnx_node_to_tensorflow_op, (ag__.ld(onnx_node), ag.ld(tensor_dict), ag.ld(self).handlers), dict(opset=ag.ld(self).opset, strict=ag__.ld(self).strict), fscope) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 439, in converted_call result = converted_f(effective_args, kwargs) File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_filekyr48t.py", line 62, in tf_onnx_node_to_tensorflow_op ag__.if_stmt(ag.ld(handlers), if_body_1, else_body_1, get_state_1, set_state_1, ('doreturn', 'retval'), 2) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 1269, in if_stmt _py_if_stmt(cond, body, orelse) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 1322, in _py_if_stmt return body() if cond else orelse() File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_filekyr48t.py", line 56, in if_body_1 ag.if_stmt(ag__.ld(handler), if_body, else_body, get_state, set_state, ('doreturn', 'retval'), 2) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 1269, in if_stmt _py_if_stmt(cond, body, orelse) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 1322, in _py_if_stmt return body() if cond else orelse() File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_filekyr48t.py", line 48, in ifbody retval = ag__.converted_call(ag.ld(handler).handle, (ag.ld(node),), dict(tensor_dict=ag__.ld(tensor_dict), strict=ag.ld(strict)), fscope) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 439, in converted_call result = converted_f(effective_args, kwargs) File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_filei8y3rukj.py", line 41, in tfhandle ag.if_stmt(ag.ld(ver_handle), if_body, else_body, get_state, set_state, ('doreturn', 'retval'), 2) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 1269, in if_stmt _py_if_stmt(cond, body, orelse) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 1322, in _py_if_stmt return body() if cond else orelse() File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_filei8y3rukj.py", line 33, in ifbody retval = ag__.converted_call(ag.ld(ver_handle), (ag__.ld(node),), dict(ag__.ld(kwargs)), fscope) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 439, in converted_call result = converted_f(effective_args, kwargs) File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_filedt4vrp2y.py", line 12, in tfversion retval_ = ag__.converted_call(ag.ld(cls)._common, (ag__.ld(node),), dict(ag.ld(kwargs)), fscope) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\impl\api.py", line 439, in converted_call result = converted_f(*effective_args, **kwargs) File "C:\Users\mdino\AppData\Local\Temp__autograph_generatedfile0e349o2q.py", line 122, in tfcommon ag.if_stmt(ag__.ld(cls).SINCE_VERSION < 11, if_body_1, else_body_1, get_state_1, set_state_1, ('constant_values', 'paddings'), 2) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 1269, in if_stmt _py_if_stmt(cond, body, orelse) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\control_flow.py", line 1322, in _py_if_stmt return body() if cond else orelse() File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_file0e349o2q.py", line 119, in else_body_1 constant_values = ag.if_exp(ag.converted_call(ag.ld(len), (ag.ld(node).inputs,), None, fscope) == 3, lambda : ag__.ld(tensor_dict)[ag.ld(node).inputs[2]], lambda : 0, 'ag.converted_call(len, (node.inputs,), None, fscope) == 3') File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\conditional_expressions.py", line 27, in if_exp return _py_if_exp(cond, if_true, if_false) File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\tensorflow\python\autograph\operators\conditional_expressions.py", line 52, in _py_if_exp return if_true() if cond else if_false() File "C:\Users\mdino\AppData\Local\Temp__autograph_generated_file0e349o2q.py", line 119, in constant_values = ag.if_exp(ag.converted_call(ag.ld(len), (ag.ld(node).inputs,), None, fscope) == 3, lambda : ag__.ld(tensor_dict)[ag.ld(node).inputs[2]], lambda : 0, 'ag__.converted_call(len, (node.inputs,), None, fscope) == 3') tensorflow.python.autograph.pyct.error_utils.MultilineMessageKeyError: in user code:

File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\onnx_tf\backend_tf_module.py", line 99, in __call__  *
    output_ops = self.backend._onnx_node_to_tensorflow_op(onnx_node,
File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\onnx_tf\backend.py", line 347, in _onnx_node_to_tensorflow_op  *
    return handler.handle(node, tensor_dict=tensor_dict, strict=strict)
File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\onnx_tf\handlers\handler.py", line 59, in handle  *
    return ver_handle(node, **kwargs)
File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\onnx_tf\handlers\backend\pad.py", line 91, in version_11  *
    return cls._common(node, **kwargs)
File "C:\Users\mdino\anaconda3\envs\mmdeploy\lib\site-packages\onnx_tf\handlers\backend\pad.py", line 73, in _common  *
    constant_values = tensor_dict[node.inputs[2]] if len(

KeyError: ''

Reproduction

Use

./tools/deploy.py configs/mmseg/segmentation_onnxruntime_static-512x512.py

to create an ONNX model from a Mobilenet_v3 model created using mmsegmentation

then run the following python commands:

import onnx from onnx_tf.backend import prepare import tensorflow as tf

Load the ONNX model

onnx_model = onnx.load(onnx_model_path)

Export TF model:

tf_rep = prepare(onnx_model) tf_rep.export_graph(tf_model_path)

Environment

PC with RTX 3090 GPU
CUDA 11.3 

absl-py==1.4.0
addict==2.4.0
aenum==3.1.15
aliyun-python-sdk-core==2.13.36
aliyun-python-sdk-kms==2.16.1
astunparse==1.6.3
brotlipy==0.7.0
cachetools==5.3.1
certifi @ file:///C:/b/abs_36eb5mzhph/croot/certifi_1690232276943/work/certifi
cffi @ file:///C:/b/abs_49n3v2hyhr/croot/cffi_1670423218144/work
charset-normalizer @ file:///tmp/build/80754af9/charset-normalizer_1630003229654/work
click==8.1.6
cloudpickle==2.2.1
colorama==0.4.6
coloredlogs==15.0.1
contourpy==1.1.0
crcmod==1.7
cryptography @ file:///C:/b/abs_13590mi9q9/croot/cryptography_1689373706078/work
cycler==0.11.0
decorator==5.1.1
dill==0.3.7
dm-tree==0.1.8
filelock @ file:///C:/b/abs_c7yrhs9uz2/croot/filelock_1672387617533/work
flatbuffers==23.5.26
fonttools==4.42.0
gast==0.4.0
google-auth==2.22.0
google-auth-oauthlib==1.0.0
google-pasta==0.2.0
grpcio==1.57.0
h5py==3.9.0
humanfriendly==10.0
idna @ file:///C:/b/abs_bdhbebrioa/croot/idna_1666125572046/work
importlib-metadata==6.8.0
importlib-resources==6.0.1
Jinja2 @ file:///C:/b/abs_7cdis66kl9/croot/jinja2_1666908141852/work
jmespath==0.10.0
keras==2.13.1
kiwisolver==1.4.4
libclang==16.0.6
Markdown==3.4.4
markdown-it-py==3.0.0
MarkupSafe @ file:///C:/ci/markupsafe_1654508077284/work
matplotlib==3.7.2
mdurl==0.1.2
mkl-fft==1.3.6
mkl-random @ file:///C:/Users/dev-admin/mkl/mkl_random_1682977971003/work
mkl-service==2.4.0
mmcls==1.0.0rc6
mmcv==2.0.0
-e git+https://github.com/open-mmlab/mmdeploy.git@a7e9be224608defa7dd8c31a7ece9146586e831d#egg=mmdeploy
mmengine==0.8.4
-e git+https://github.com/open-mmlab/mmsegmentation.git@30a3f94f3e2916e27fa38c67cc3b8c69c1893fe8#egg=mmsegmentation
model-index==0.1.11
modelindex==0.0.2
mpmath @ file:///C:/b/abs_7833jrbiox/croot/mpmath_1690848321154/work
multiprocess==0.70.15
networkx @ file:///C:/b/abs_e6gi1go5op/croot/networkx_1690562046966/work
numpy==1.24.3
oauthlib==3.2.2
onnx==1.14.0
onnx-tf==1.10.0
onnxruntime==1.15.1
opencv-python==4.8.0.76
opendatalab==0.0.10
openmim==0.3.9
openxlab==0.0.17
opt-einsum==3.3.0
ordered-set==4.1.0
oss2==2.17.0
packaging==23.1
pandas==2.0.3
Pillow==9.4.0
platformdirs==3.10.0
prettytable==3.8.0
protobuf==4.24.0
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work
pycryptodome==3.18.0
Pygments==2.16.1
pyOpenSSL @ file:///C:/b/abs_08f38zyck4/croot/pyopenssl_1690225407403/work
pyparsing==3.0.9
pyreadline3==3.4.1
PySocks @ file:///C:/ci/pysocks_1605307512533/work
python-dateutil==2.8.2
pytz==2023.3
pywin32==306
PyYAML==6.0.1
regex==2023.8.8
requests==2.28.2
requests-oauthlib==1.3.1
rich==13.4.2
rsa==4.9
scipy==1.11.1
six==1.16.0
sympy @ file:///C:/b/abs_95fbf1z7n6/croot/sympy_1668202411612/work
tabulate==0.9.0
tensorboard==2.13.0
tensorboard-data-server==0.7.1
tensorflow==2.13.0
tensorflow-addons==0.21.0
tensorflow-estimator==2.13.0
tensorflow-intel==2.13.0
tensorflow-io-gcs-filesystem==0.31.0
tensorflow-probability==0.21.0
termcolor==2.3.0
terminaltables==3.1.10
tflite==2.10.0
tomli==2.0.1
torch==2.0.1
torchaudio==2.0.2
torchvision==0.15.2
tqdm==4.65.2
typeguard==2.13.3
typing_extensions==4.5.0
tzdata==2023.3
urllib3 @ file:///C:/b/abs_889_loyqv4/croot/urllib3_1686163174463/work
wcwidth==0.2.6
Werkzeug==2.3.7
win-inet-pton @ file:///C:/ci/win_inet_pton_1605306162074/work
wrapt==1.15.0
yapf==0.40.1
zipp==3.16.2

Error traceback

No response

RunningLeon commented 1 year ago

hi, mmdeploy does not support onnx2tf. It seems something wrong when parsing Pad node. Maybe you could try upgrade opset_version.

github-actions[bot] commented 1 year ago

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

dino-illumix commented 1 year ago

What's the correct way to create TFLite models from open_mmlab models if not by using onnx2tf? There does not appear to be any way to directly create either TFLite models or Tensorflow models that can be converted to TFLite.

dino-illumix commented 1 year ago

I tried ugrading opset version. The highest version which would run onnx2tf was v17 but this still gives the same keyerror. I'd really like to build this MobilenetV3 model but it really needs to be in tflite format (that's the whole point of building a Mobilenet model as I need it to be small and run on device).

github-actions[bot] commented 1 year ago

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

github-actions[bot] commented 1 year ago

This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now.

open-mmlab / mmdeploy

Mobilenet_v3 mmseg models converted to ONNX fail when attempting conversion to Tensorflow models #2417

Checklist

Describe the bug

Load the ONNX model

Export TF model:

Reproduction

Load the ONNX model

Export TF model:

Environment

Error traceback