search

PINTO0309 / openvino2tensorflow

This script converts the ONNX/OpenVINO IR model to Tensorflow's saved_model, tflite, h5, tfjs, tftrt(TensorRT), CoreML, EdgeTPU, ONNX and pb. PyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW). And the conversion from .pb to saved_model and from saved_model to .pb and from .pb to .tflite and saved_model to .tflite and saved_model to onnx. Support for building environments with Docker. It is possible to directly access the host PC GUI and the camera to verify the operation. NVIDIA GPU (dGPU) support. Intel iHD GPU (iGPU) support.
MIT License
334 stars 40 forks source link

Use docker to convert the sample onnx model failed #112

Closed liyancas closed 2 years ago

liyancas commented 2 years ago

Issue Type

Bug

OS

Ubuntu

OS architecture

x86_64

Programming Language

Python

Framework

OpenVINO

Download URL for ONNX / OpenVINO IR

https://digital-standard.com/threedpose/models/Resnet34_3inputs_448x448_20200609.onnx

Convert Script

simplified_onnx_model, success = onnxsim.simplify(onnx_model_path)

assert success, 'Failed to simplify the ONNX model. You may have to skip this step' print(f'Generating {simplified_onnx_model_path} ...') onnx.save(simplified_onnx_model, simplified_onnx_model_path)

mo --input_model /work/model_convert/test/onnx/Resnet34_3inputs_448x448_20200609.simple.onnx --input_shape [1,3,448,448] --output_dir /work/model_convert/test/openvino --data_type FP32

Description

Relevant Log Output

root@18ea9b28075f:/work/model_convert/test# python batch_convert_tflite.py
2022-07-06 08:50:33.819606: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2022-07-06 08:50:34.735010: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
Resnet34_3inputs_448x448_20200609.onnx
2022-07-06 08:50:36.112232: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: /usr/lib/x86_64-linux-gnu/libcuda.so.1: file too short
2022-07-06 08:50:36.112269: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2022-07-06 08:50:36.112300: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:163] no NVIDIA GPU device is present: /dev/nvidia0 does not exist
2022-07-06 08:50:36.112708: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE3 SSE4.1 SSE4.2 AVX AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
[1,3,448,448]
/work/model_convert/test/openvino
Generating openvino at: /work/model_convert/test/openvino
mo --input_model /work/model_convert/test/onnx/Resnet34_3inputs_448x448_20200609.simple.onnx --input_shape [1,3,448,448] --output_dir /work/model_convert/test/openvino --data_type FP32
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.23ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:  /work/model_convert/test/onnx/Resnet34_3inputs_448x448_20200609.simple.onnx
    - Path for generated IR:    /work/model_convert/test/openvino
    - IR output name:   Resnet34_3inputs_448x448_20200609.simple
    - Log level:    ERROR
    - Batch:    Not specified, inherited from the model
    - Input layers:     Not specified, inherited from the model
    - Output layers:    Not specified, inherited from the model
    - Input shapes:     [1,3,448,448]
    - Source layout:    Not specified
    - Target layout:    Not specified
    - Layout:   Not specified
    - Mean values:  Not specified
    - Scale values:     Not specified
    - Scale factor:     Not specified
    - Precision of IR:  FP32
    - Enable fusing:    True
    - User transformations:     Not specified
    - Reverse input channels:   False
    - Enable IR generation for fixed input shape:   False
    - Use the transformations config file:  None
Advanced parameters:
    - Force the usage of legacy Frontend of Model Optimizer for model conversion into IR:   False
    - Force the usage of new Frontend of Model Optimizer for model conversion into IR:  False
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.23ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
OpenVINO runtime found in:  /usr/local/lib/python3.8/dist-packages/openvino
OpenVINO runtime version:   custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
Model Optimizer version:    custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.23ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
[ WARNING ]
Detected not satisfied dependencies:
    numpy: installed: 1.23.0, required: < 1.20

Please install required versions of components or run pip installation
pip install openvino-dev
[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]
[ ERROR ]  Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/openvino/tools/mo/main.py", line 533, in main
    ret_code = driver(argv)
  File "/usr/local/lib/python3.8/dist-packages/openvino/tools/mo/main.py", line 489, in driver
    graph, ngraph_function = prepare_ir(argv)
  File "/usr/local/lib/python3.8/dist-packages/openvino/tools/mo/main.py", line 394, in prepare_ir
    ngraph_function = moc_pipeline(argv, moc_front_end)
  File "/usr/local/lib/python3.8/dist-packages/openvino/tools/mo/moc_frontend/pipeline.py", line 31, in moc_pipeline
    user_shapes, outputs, freeze_placeholder = fe_user_data_repack(
  File "/usr/local/lib/python3.8/dist-packages/openvino/tools/mo/moc_frontend/extractor.py", line 221, in fe_user_data_repack
    _input_shapes, _freeze_placeholder = fe_input_user_data_repack(
  File "/usr/local/lib/python3.8/dist-packages/openvino/tools/mo/moc_frontend/extractor.py", line 169, in fe_input_user_data_repack
    assert len(model_inputs) == 1
AssertionError

[ ERROR ]  ---------------- END OF BUG REPORT --------------
[ ERROR ]  -------------------------------------------------

Source code for simple inference testing code

No response

PINTO0309 commented 2 years ago 
$ onnxsim Resnet34_3inputs_448x448_20200609.onnx Resnet34_3inputs_448x448_20200609.onnx 
Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃                    ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Add                │ 18             │ 18               │
│ BatchNormalization │ 56             │ 3                │
│ Concat             │ 2              │ 2                │
│ Conv               │ 56             │ 56               │
│ ConvTranspose      │ 1              │ 1                │
│ MaxPool            │ 3              │ 3                │
│ Relu               │ 53             │ 53               │
│ Tanh               │ 2              │ 2                │
│ Model Size         │ 108.7MiB       │ 108.5MiB         │
└────────────────────┴────────────────┴──────────────────┘

$ mo \
--framework onnx \
--input_model Resnet34_3inputs_448x448_20200609.onnx \
--data_type FP32 \
--output_dir Resnet34_3inputs_448x448_20200609/openvino/FP32 \
--model_name Resnet34_3inputs_448x448_20200609

Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:  /home/user/workdir/Resnet34_3inputs_448x448_20200609.onnx
    - Path for generated IR:    /home/user/workdir/Resnet34_3inputs_448x448_20200609/openvino/FP32
    - IR output name:   Resnet34_3inputs_448x448_20200609
    - Log level:    ERROR
    - Batch:    Not specified, inherited from the model
    - Input layers:     Not specified, inherited from the model
    - Output layers:    Not specified, inherited from the model
    - Input shapes:     Not specified, inherited from the model
    - Source layout:    Not specified
    - Target layout:    Not specified
    - Layout:   Not specified
    - Mean values:  Not specified
    - Scale values:     Not specified
    - Scale factor:     Not specified
    - Precision of IR:  FP32
    - Enable fusing:    True
    - User transformations:     Not specified
    - Reverse input channels:   False
    - Enable IR generation for fixed input shape:   False
    - Use the transformations config file:  None
Advanced parameters:
    - Force the usage of legacy Frontend of Model Optimizer for model conversion into IR:   False
    - Force the usage of new Frontend of Model Optimizer for model conversion into IR:  False
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.23ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
OpenVINO runtime found in:  /usr/local/lib/python3.8/dist-packages/openvino
OpenVINO runtime version:   custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
Model Optimizer version:    custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ WARNING ]  
Detected not satisfied dependencies:
    numpy: installed: 1.22.4, required: < 1.20

Please install required versions of components or run pip installation
pip install openvino-dev
[ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /home/user/workdir/Resnet34_3inputs_448x448_20200609/openvino/FP32/Resnet34_3inputs_448x448_20200609.xml
[ SUCCESS ] BIN file: /home/user/workdir/Resnet34_3inputs_448x448_20200609/openvino/FP32/Resnet34_3inputs_448x448_20200609.bin
[ SUCCESS ] Total execution time: 0.56 seconds. 
[ SUCCESS ] Memory consumed: 305 MB. 
It's been a while, check for a new version of Intel(R) Distribution of OpenVINO(TM) toolkit here https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit/download.html?cid=other&source=prod&campid=ww_2022_bu_IOTG_OpenVINO-2022-1&content=upg_all&medium=organic or on the GitHub*
[ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11.
Find more information about API v2.0 and IR v11 at https://docs.openvino.ai
liyancas commented 2 years ago

When I deleted the input shape parameter of mo, it succeed. Do you know what's the reason?

PINTO0309 commented 2 years ago

Instructions on how to use OpenVINO are not supported by this repository. Read the OpenVINO tutorial or submit an issue in the OpenVINO repository.

mo \
--framework onnx \
--input_model Resnet34_3inputs_448x448_20200609.onnx \
--input input.1,input.4,input.7 \
--input_shape [1,3,448,448],[1,3,448,448],[1,3,448,448] \
--data_type FP32 \
--output_dir Resnet34_3inputs_448x448_20200609/openvino/FP32 \
--model_name Resnet34_3inputs_448x448_20200609

Model Optimizer arguments:
Common parameters:
    - Path to the Input Model:  /home/user/workdir/Resnet34_3inputs_448x448_20200609.onnx
    - Path for generated IR:    /home/user/workdir/Resnet34_3inputs_448x448_20200609/openvino/FP32
    - IR output name:   Resnet34_3inputs_448x448_20200609
    - Log level:    ERROR
    - Batch:    Not specified, inherited from the model
    - Input layers:     input.1,input.4,input.7
    - Output layers:    Not specified, inherited from the model
    - Input shapes:     [1,3,448,448],[1,3,448,448],[1,3,448,448]
    - Source layout:    Not specified
    - Target layout:    Not specified
    - Layout:   Not specified
    - Mean values:  Not specified
    - Scale values:     Not specified
    - Scale factor:     Not specified
    - Precision of IR:  FP32
    - Enable fusing:    True
    - User transformations:     Not specified
    - Reverse input channels:   False
    - Enable IR generation for fixed input shape:   False
    - Use the transformations config file:  None
Advanced parameters:
    - Force the usage of legacy Frontend of Model Optimizer for model conversion into IR:   False
    - Force the usage of new Frontend of Model Optimizer for model conversion into IR:  False
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.23ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
OpenVINO runtime found in:  /usr/local/lib/python3.8/dist-packages/openvino
OpenVINO runtime version:   custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
Model Optimizer version:    custom_HEAD_cdb9bec7210f8c24fde3e416c7ada820faaaa23e
[ WARNING ]  
Detected not satisfied dependencies:
    numpy: installed: 1.22.4, required: < 1.20

Please install required versions of components or run pip installation
pip install openvino-dev
[ SUCCESS ] Generated IR version 11 model.
[ SUCCESS ] XML file: /home/user/workdir/Resnet34_3inputs_448x448_20200609/openvino/FP32/Resnet34_3inputs_448x448_20200609.xml
[ SUCCESS ] BIN file: /home/user/workdir/Resnet34_3inputs_448x448_20200609/openvino/FP32/Resnet34_3inputs_448x448_20200609.bin
[ SUCCESS ] Total execution time: 0.64 seconds. 
[ SUCCESS ] Memory consumed: 306 MB. 
It's been a while, check for a new version of Intel(R) Distribution of OpenVINO(TM) toolkit here https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit/download.html?cid=other&source=prod&campid=ww_2022_bu_IOTG_OpenVINO-2022-1&content=upg_all&medium=organic or on the GitHub*
[ INFO ] The model was converted to IR v11, the latest model format that corresponds to the source DL framework input/output format. While IR v11 is backwards compatible with OpenVINO Inference Engine API v1.0, please use API v2.0 (as of 2022.1) to take advantage of the latest improvements in IR v11.
Find more information about API v2.0 and IR v11 at https://docs.openvino.ai