Closed lee-man closed 3 years ago
lee-man
commented
3 years ago Also, in Google's official Movenet model, the Conv + ReLU6, and Conv +ReLU are fused together. Also the upsampling op: ResizeBilinear is used. Do you have any idea to use these specific ops when we convert the PyTorch model to TFLite for better performance? Thanks! The model definition of Pytorch Movenet is included here: https://github.com/lee-man/movenet-pytorch/blob/main/movenet/models/movenet.py
PINTO0309
commented
3 years ago The custom model you have created is quite complex because it contains many layers of Transpose, Reshape, Gather, etc... that are less than 4D after the middle of the model. Therefore, it is not possible to mechanically convert NCHW format to NHWC format programmatically.
If the layer contains less than 4D shape changes, you will need to customize the tool's behavior.
PyTorch MoveNet to tflite, pb, saved_model, xml/bin https://drive.google.com/file/d/1z_qxyI9-kBaDYUCoWSbsu56Mw9vvb55k/view?usp=sharing
replace.json
{
"format_version": 2,
"layers": [
{
"layer_id": "364",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
1,
2
]
},
{
"layer_id": "366",
"type": "Const",
"replace_mode": "direct",
"values": [
48,
2,
48,
17
]
},
{
"layer_id": "393",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
1,
2
]
},
{
"layer_id": "412",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
1,
2
]
},
{
"layer_id": "438",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
1,
2
]
},
{
"layer_id": "443",
"type": "Const",
"replace_mode": "direct",
"values": [
1,
1,
2304
]
},
{
"layer_id": "499",
"type": "Const",
"replace_mode": "direct",
"values": [
1,
17,
2304
]
},
{
"layer_id": "526",
"type": "Reshape",
"replace_mode": "insert_after",
"values": [
1,
17
]
},
{
"layer_id": "533",
"type": "Reshape",
"replace_mode": "insert_after",
"values": [
1,
17
]
},
{
"layer_id": "537",
"type": "Reshape",
"replace_mode": "insert_after",
"values": [
17
]
},
{
"layer_id": "556",
"type": "Reshape",
"replace_mode": "insert_after",
"values": [
1,
17
]
},
{
"layer_id": "563",
"type": "Reshape",
"replace_mode": "insert_after",
"values": [
1,
17
]
},
{
"layer_id": "568",
"type": "Const",
"replace_mode": "direct",
"values": [
0
]
},
{
"layer_id": "600",
"type": "Const",
"replace_mode": "direct",
"values": [
2
]
},
{
"layer_id": "602",
"type": "Const",
"replace_mode": "direct",
"values": [
2
]
}
]
}python3 openvino2tensorflow_special_custom.py \
--model_path movenet_opt.xml \
--output_saved_model --output_pb \
--output_no_quant_float32_tflite \
--weight_replacement_config replace.jsonI followed through to the final layer to make sure it was converted correctly, but I have not checked to see if the inference works correctly. I was only trying to show you that the conversion would be successful. If the inference results do not meet your expectations, please adjust the resources I have attached here yourself.
Also, in Google's official Movenet model, the Conv + ReLU6, and Conv +ReLU are fused together. Also the upsampling op: ResizeBilinear is used. Do you have any idea to use these specific ops when we convert the PyTorch model to TFLite for better performance?
I have no idea what your concern is. Sorry. :crying_cat_face:
Let me ask you one last question. Why did you implement it in PyTorch and not in TensorFlow? The cost is too high. :sweat_smile:
lee-man
commented
3 years ago @PINTO0309 Thanks for your quick response! I really appreciate it and will check it later.
I have no idea what your concern is. Sorry. 😿
I thought that the Conv layer and ReLU will not be fused together. It's my fault.
Let me ask you one last question. Why did you implement it in PyTorch and not in TensorFlow? The cost is too high. 😅
Actually, my first step is trying to reimplement the Movenet structure using PyTorch and train it on my own dataset. However, it turned out that the performance is not that satisfactory. So I decided to reimplement the inference part and try to make it the same as the original one. Indeed the cost is too high as you said.
Min
1. OS you are using MacOS
2. OS Architecture e.g. x86_64, armv7l, aarch64, etc
3. Version of OpenVINO e.g. 2021.2.185, etc
4. Version of TensorFlow e.g. v2.4.1, tf-nightly==2.5.0.dev20210128, etc
5. Version of TensorRT e.g. TensorRT6.0 GA, etc
6. Version of TFJS e.g. 1.5.0, etc
7. Version of coremltools e.g. 4.0, etc
8. Version of ONNX e.g. v1.8.0, etc
9. Download URL for ONNX model
10. Download URL for OpenVINO IR (.bin/.xml) model
11. URL of the repository from which the transformed model was taken
12. URL or source code for simple inference testing code
13. Issue Details
I try to convert the Pytorch implementation of Movenet to TFLite with the correct ordering of channels. However, when I try to convert the obtained openvino model to TFLite, there's an error of incorrect input shape. I check your doc: https://github.com/PINTO0309/openvino2tensorflow#6-7-replace-weights-or-constant-values-in-const-op-and-add-transpose-or-reshape-just-before-the-operation-specified-by-layer_id , and still have no clue how to fix the problem. Could you help check it out? Thanks a lot. The following is the log:
And I attached the corresponding model files here. openvino.zip
Best wishes, Min