The BN layer disappears when U2NET is converted

zhenzhen1003 commented 3 years ago

1. WIndows10

2. x86_64

3. Version of OpenVINO 2021.4.582

4. Version of TensorFlow @2.6.0rc1

5. Version of ONNX v1.8.0

6. Issue Details

I have an U2net model in ONNX format, in which the BN layer is BatchNormlization, which is converted into the IR model of OpenVino and then fused into the convolution operation. When converted into TFLite through OpenVino2TensorFlow, only the convolution layer is left, and the BN layer is not transformed. Such model prediction is very inaccurate. What should I do? Thanks for your answer.

zhenzhen1003 commented 3 years ago

This is the ONNX model

This is the IR model

This is tensorflow model

PINTO0309 commented 3 years ago

During the optimization process, the BN is merged into Conv2D or GroupConvolution or DepthwiseConv2D Weights. This is not a particular problem. If you are concerned about this, you can calculate the weights by hand, although it is tedious. My tool does extreme optimization. Rather, the overall performance of the model has been improved.

If you want to know the specifications of BN fusion, please see below. https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimization_Techniques.html

If you have a problem with BatchNormalization disappearing, you can add a layer corresponding to Normalization to the .xml file yourself. BN fusion is performed by OpenVINO Model Optimizer, not by openvino2tensorflow.

You need to verify if the degradation in accuracy you describe is really due to BN fusion. How much difference is there in the output value right after ReLU?

I don't know how detailed your research actually was, so I could only give you this level of response at this point.

Have you read the following document? Have you not read them? I have no idea what you are reading and understanding, so I have to ask questions about everything.

Converting a Model Using General Conversion Parameters https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model_General.html

  --disable_fusing      Turn off fusing of linear operations to Convolution
  --disable_resnet_optimization
                        Turn off resnet optimization
  --finegrain_fusing FINEGRAIN_FUSING
                        Regex for layers/operations that won't be fused.
                        Example: --finegrain_fusing Convolution1,.*Scale.*
  --disable_gfusing     Turn off fusing of grouped convolutions

The following tools are very useful when debugging a split model. https://github.com/yas-sim/openvino-model-division-and-simple-custom-layer

I am very sorry if I have made you feel uncomfortable. But I always find that many people's questions are so fragmented that I don't always understand what they really mean. And for that research I spend hours on one question.

zhenzhen1003 commented 3 years ago

During the optimization process, the BN is merged into Conv2D or GroupConvolution or DepthwiseConv2D Weights. This is not a particular problem. If you are concerned about this, you can calculate the weights by hand, although it is tedious. My tool does extreme optimization. Rather, the overall performance of the model has been improved.

If you want to know the specifications of BN fusion, please see below. https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimization_Techniques.html

If you have a problem with BatchNormalization disappearing, you can add a layer corresponding to Normalization to the .xml file yourself. BN fusion is performed by OpenVINO Model Optimizer, not by openvino2tensorflow.

You need to verify if the degradation in accuracy you describe is really due to BN fusion. How much difference is there in the output value right after ReLU?

I don't know how detailed your research actually was, so I could only give you this level of response at this point.

Have you read the following document? Have you not read them? I have no idea what you are reading and understanding, so I have to ask questions about everything.

Converting a Model Using General Conversion Parameters https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model_General.html
  --disable_fusing      Turn off fusing of linear operations to Convolution
  --disable_resnet_optimization
                        Turn off resnet optimization
  --finegrain_fusing FINEGRAIN_FUSING
                        Regex for layers/operations that won't be fused.
                        Example: --finegrain_fusing Convolution1,.*Scale.*
  --disable_gfusing     Turn off fusing of grouped convolutions
The following tools are very useful when debugging a split model. https://github.com/yas-sim/openvino-model-division-and-simple-custom-layer

I am very sorry if I have made you feel uncomfortable. But I always find that many people's questions are so fragmented that I don't always understand what they really mean. And for that research I spend hours on one question.

Thank you very much. Your answer is very useful to me, now I have understood the operation of BN layer fusion, thank you again

PINTO0309 commented 3 years ago

Closed due to lack of progress.

PINTO0309 / openvino2tensorflow