Onnx to NNEF for Retinanet - Error Float list tensors must be constant to be evaluable for now

[KarthikDutt]

I am trying to convert a retinanet onnx model to nnef as a bridge to finally convert it to tflite. The model was originally trained in pytorch and i used onnx to convert the pytorch model to onnx. The following command was used to convert the onnx model to nnef:

convert.py --input-format onnx --output-format nnef --input-model retinanet.onnx --output-model retinanetnnef.tgz --input-shape="[1, 3, 256, 256]" --compress

The error is as shown below:

Traceback (most recent call last): File "convert.py", line 699, in convert_using_argv(sys.argv) File "convert.py", line 687, in convert_using_argv conversion_info=args.conversion_info) File "convert.py", line 485, in convert custom_converters=custom_converters)) File "convert.py", line 376, in convert_using_premade_objects source_graph = reader(*in_filename) File "C:\Users\kar11081\Anaconda3\envs\geosaurus_dev_env_kar\lib\site-packages\nnef_tools-0.1-py3.6.egg\nnef_tools\io\onnx\onnx_io.py", line 533, in call onnx_shape_inference.infer_shapes(g, source_shapes=self._input_shape, custom_shapes=self._custom_shapes) File "C:\Users\kar11081\Anaconda3\envs\geosaurus_dev_env_kar\lib\site-packages\nnef_tools-0.1-py3.6.egg\nnef_tools\io\onnx\onnx_shape_inference.py", line 49, in infer_shapes inferred_shapes, inferred_dtypes = shape_functionsop.name File "C:\Users\kar11081\Anaconda3\envs\geosaurus_dev_env_kar\lib\site-packages\nnef_tools-0.1-py3.6.egg\nnef_tools\io\onnx\onnx_shape_inference.py", line 587, in propagate_upsample op.attribs['scales'] = evaluate_float_list_tensor_simple(op.inputs[1]) File "C:\Users\kar11081\Anaconda3\envs\geosaurus_dev_env_kar\lib\site-packages\nnef_tools-0.1-py3.6.egg\nnef_tools\io\onnx\onnx_shape_inference.py", line 158, in evaluate_float_list_tensor_simple assert False, "Float list tensors must be constant to be evaluable for now." AssertionError: Float list tensors must be constant to be evaluable for now.

The versions of ONNX that i am using is 1.7. Pytorch version is 1.2

Steps to reproduce the error: Execute the command displayed above which caused the error. The onnx model file can be downloaded from

https://drive.google.com/file/d/1q5e1bK9GcK1ZgkQhHrttdTVm1qhqbd9a/view?usp=sharing

[gyenesvi]

It seems that your ONNX model contains dynamic tensors shapes, and NNEF only supports static shapes. The converter attempts to evaluate such tensor shapes if they happen to be constants, but it seems that it fails here; it does not have full coverage in that respect. I suggest you try this tool on the ONNX model before conversion: https://github.com/daquexian/onnx-simplifier. That could fix the ONNX to NNEF conversion step. However, I also see that your model contains batch-normalization ops, and tflite does not support that, and the NNEF to TFLite converter is not yet able to optimize it away.

We are working on an internal restructuring of the converters that should solve both problems, however, it needs some time for verification before we can push this update. I have tried to convert your model with my latest code locally, and after some bug fixing it seems to have worked, but the resulting NNEF and TFLite model still contains some unnecessary operations that needs optimization. Hopefully we can push this update soon and you will be able to convert the model.

[KarthikDutt]

Thanks for the update and great work. Will look forward to the update.

[gyenesvi]

The NNEF tools have been updated recently. When converting from ONNX, if you turn on the --fold-constants flag in convert.py, the converter attempts to evaluate tensor shapes if they happen to be constants (runs, the above onnx-simplifier tool, so you need to have that installed).

Furthermore, if you turn on the --optimize flag, the resulting NNEF will be optimized, merging a bunch of ops, like batch norm into conv. After that, converting to TFLite should work. So this should solve your issues hopefully. Let me know!