Closed fricc33 closed 2 years ago
GroupConvolution
bug fixes. a1a0c18cd6b1e7002c8ebf3f220c075cc2d0682c
This tool has been upgraded to v1.22.2. BTW, Docker works on Mac as well, so using a Docker container is highly recommended.
cd tf
wget -O ../weights/midas_v21-f6b98070.pt \
https://github.com/AlexeyAB/MiDaS/releases/download/midas_dpt/midas_v21-f6b98070.pt
python make_onnx_model.py
python3 -m onnxsim midas_v21-f6b98070.onnx midas_v21-f6b98070.onnx
docker pull pinto0309/openvino2tensorflow:latest
docker run -it --rm \
-v `pwd`:/home/user/workdir \
pinto0309/openvino2tensorflow:latest
cd workdir
$INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo.py \
--input_model midas_v21-f6b98070.onnx \
--data_type FP32 \
--output_dir openvino/FP32
openvino2tensorflow \
--model_path openvino/FP32/midas_v21-f6b98070.xml \
--model_output_path openvino-tflite \
--output_no_quant_float32_tflite
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 768, 1024, 1), dtype=tf.float32, name=None), name='tf.nn.relu_115/Relu:0', description="created by layer 'tf.nn.relu_115'")
====================================================================================
layer_type: Const
layer_id: 728
tf_layers_dict: (1,)
====================================================================================
layer_type: Squeeze
layer_id: 729
input_layer0: layer_id=727: KerasTensor(type_spec=TensorSpec(shape=(1, 768, 1024, 1), dtype=tf.float32, name=None), name='tf.nn.relu_115/Relu:0', description="created by layer 'tf.nn.relu_115'")
input_layer1: layer_id=728: Const(ndarray).shape (1,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 768, 1024), dtype=tf.float32, name=None), name='tf.compat.v1.squeeze/Squeeze:0', description="created by layer 'tf.compat.v1.squeeze'")
====================================================================================
layer_type: Result
layer_id: 730
input_layer0: layer_id=729: KerasTensor(type_spec=TensorSpec(shape=(1, 768, 1024), dtype=tf.float32, name=None), name='tf.compat.v1.squeeze/Squeeze:0', description="created by layer 'tf.compat.v1.squeeze'")
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 768, 1024), dtype=tf.float32, name=None), name='tf.identity/Identity:0', description="created by layer 'tf.identity'")
====================================================================================
TensorFlow/Keras model building process complete!
tflite Float32 convertion started ===================================================
tflite Float32 convertion complete! - openvino-tflite/model_float32.tflite
All the conversion process is finished! =============================================
MiDaS uses a lot of GroupConvolution
in the depth direction, so I apply a workaround to decompose the converted tflite into Conv2D
. This is because TensorFlow Lite does not support GroupConvolution
. If you don't like the structure of the model, use onnx-tensorflow. However, onnx-tensorflow
does not support clean conversion from NCHW to NHWC.
Awesome! Thank you so much for your help and for your great tool.
I had actually figured out myself as well what the issue was with the padding not being consumed.
There is one more issue with MaxPool, which also allows for padding, see attached patch for a fix. Now midasnet generated with openvino2tensorflow produces identical results to onnx-tensorflow, and the resulting model is twice as fast 😎
Thanks again, cheers,
LGTM.
MaxPool
bug fixes. c07667b35e0a6dbe9bd4d1a3dc4024dd862cc38f
This tool has been upgraded to v1.22.3.
The problem of degraded accuracy of MiDaS, which had been a problem for a long time, may have been resolved. Thank you. https://github.com/PINTO0309/PINTO_model_zoo/issues/15
I would like to know what you can tell me in order to share your knowledge with other engineers. Can you tell me the inference time for the model generated by onnx-tensorflow and the approximate inference time for the model generated by openvino2tensorflow?
Great, thanks! BTW: I used to work in the TfLite GPU support team at Google Research, maybe I can give grouped convolutions a crack at a certain point. I run my model on the GPU so I would prioritize that. Maybe we can close this issue for now :)
1. OS you are using: macOS
2. OS Architecture: Intel
3. Version of OpenVINO: openvino_2021.4.689
4. Version of TensorFlow: 2.6.0
5. Version of TensorRT: NA
6. Version of TFJS: NA
7. Version of coremltools NA
8. Version of ONNX: 1.10.1
9. Download URL for ONNX model: generated
10. Download URL for OpenVINO IR: generated
11. URL of the repository from which the transformed model was taken: https://github.com/isl-org/MiDaS
12. URL or source code for simple inference testing code: see attached zip file
13. Issue Details
I'd like to try this again as a follow up to https://github.com/PINTO0309/openvino2tensorflow/issues/74, I apologize for my lack of clarity in my previous submission. My original message is at the bottom.
I tried processing your image in my environment and it worked just fine:
Maybe the issue is with the model weights file? I downloaded mine (MiDaS v2.1 large) from:
https://github.com/AlexeyAB/MiDaS/releases/download/midas_dpt/midas_v21-f6b98070.pt
Following the link on https://github.com/isl-org/MiDaS
Please let me know how we can work together on this.
Thank you very much for your patience and help,
openvino2tensorflow crashes with a tensor size mismatch to the input of layer Add 24.
I could trace the issue to the convolution and group convolution pad management logic (lines 626 and 1687) Somehow the padding in the group convolution causes the output tensor to shrink by 2 pixels in both dimensions.
I don't have a fix, except that commenting out the code that introduces the padding allows the model to be generated, although the model will produce erroneous results.
I attach a zip file with the MiDaS distribution with all the ingredients to reproduce the issue: Run the openvino2tensorflow_crash.sh script in the unzipped directory. You will have to populate the weights directory with midas_v21-f6b98070.pt, and modify the script with the path to your OpenVINO distribution.
Midas-vino-crash.zip