YoloV4 broken with OpenVINO 2021.1

at1993 commented 3 years ago

Hello,

It seems OpenVINO 2021.1 has broken the full yolov4 converted from this tool. The tiny model is still ok. Anyone find similar result?

TNTWEN commented 3 years ago

@at1993 I haven't tested it on the latest OpenVINO.I will have a try. You can also provide your error message. And you could try this repos first https://github.com/TNTWEN/OpenVINO-YOLO-Automatic-Generation and see if this project can be used.

TNTWEN commented 3 years ago

@at1993 I have test it. FULL YOLOV4

2020.4 yolov3demo +2020.4 Model Optimizer+2021.1 inference engine +CPU/GPU work well
2020.4 yolov3demo +2021.1 Model Optimizer+2021.1 inference engine +CPU error like this:

[ ERROR ] Following layers are not supported by the plugin for specified device CPU: Exp_28315, Add_28317, Exp_28319, Add_28321, Exp_28323, Add_28325, Exp_28327, Add_28329, Exp_28331, Add_28333, Exp_28335, Add_28337, Exp_28339, Add_28341, Exp_28343, Add_28345, Exp_28347, Add_28349, Exp_28375, Add_28377, Exp_28351, Add_28353, Exp_28355, Add_28357, Exp_28359, Add_28361, Exp_28363, Add_28365, Exp_28367, Add_28369, Exp_28371, Add_28373, Exp_28379, Add_28381, Exp_28383, Add_28385, Exp_28387, Add_28389, Exp_28391, Add_28393, Exp_28395, Add_28397, Exp_28399, Add_28401, Exp_28403, Add_28405, Exp_28407, Add_28409, Exp_28411, Add_28413, Exp_28415, Add_28417, Exp_28419, Add_28421, Exp_28423, Add_28425, Exp_28427, Add_28429, Exp_28431, Add_28433, Exp_28435, Add_28437, Exp_28439, Add_28441, Exp_28443, Add_28445, Exp_28447, Add_28449, Exp_28451, Add_28453, Exp_28455, Add_28457, Exp_28459, Add_28461, Exp_28463, Add_28465, Exp_28467, Add_28469, Exp_28471, Add_28473, Exp_28475, Add_28477, Exp_28479, Add_28481, Exp_28483, Add_28485, Exp_28487, Add_28489, Exp_28491, Add_28493, Exp_28495, Add_28497, Exp_28499, Add_28501, Exp_28503, Add_28505, Exp_28507, Add_28509, Exp_28511, Add_28513, Exp_28515, Add_28517, Exp_28519, Add_28521, Exp_28523, Add_28525, Exp_28527, Add_28529, Exp_28531, Add_28533, Exp_28535, Add_28537, Exp_28539, Add_28541, Exp_28543, Add_28545, Exp_28547, Add_28549, Exp_28551, Add_28553, Exp_28555, Add_28557, Exp_28559, Add_28561, Exp_28563, Add_28565, Exp_28567, Add_28569, Exp_28571, Add_28573, Exp_28575, Add_28577, Exp_28579, Add_28581, Exp_28583, Add_28585, Exp_28587, Add_28589, Exp_28591, Add_28593, Exp_28595, Add_28597, Exp_28599, Add_28601 [ ERROR ] Please try to specify cpu extensions library path in sample's command line parameters using -l or --cpu_extension command line argument

2020.4 yolov3demo +2021.1 Model Optimizer+2021.1 inference engine +GPU work well 2021.1 yolov3demo +2021.1 Model Optimizer+2021.1 inference engine +CPU the program is stuck and no output 2021.1 yolov3demo +2021.1 Model Optimizer+2021.1 inference engine +GPU work well

So if we all use 2021.1 Only the CPU is stuck, because there is no error feedback, I need further testing

TNTWEN commented 3 years ago

All 2021.1 ,YOLOv4 tiny version works well

TNTWEN commented 3 years ago

@at1993 https://github.com/TNTWEN/OpenVINO-YOLO-Automatic-Generation use this repos ,it work well!!

You could replace https://github.com/TNTWEN/OpenVINO-YOLOV4/blob/master/yolo_v4.py with https://github.com/TNTWEN/OpenVINO-YOLO-Automatic-Generation/blob/master/demo/yolo_v4.py

Please have a try ,and tell me if it works! If it works I will combine the two repos！！

TNTWEN commented 3 years ago

I have add OpenVINO 2021.1 support ：https://github.com/TNTWEN/OpenVINO-YOLOV4/tree/OpenVINO2021.1

at1993 commented 3 years ago

@TNTWEN Thank you so much for your effort. My tests results are as follow:

2021.1 yolov3demo +2021.1 Model Optimizer+2021.1 inference engine --data_type FP32 CPU/GPU - OK! 2021.1 yolov3demo +2021.1 Model Optimizer+2021.1 inference engine --data_type FP16 CPU - OK! 2021.1 yolov3demo +2021.1 Model Optimizer+2021.1 inference engine --data_type FP16 GPU - Failed - No bounding boxes!

Sorry @TNTWEN I should have been more clear regarding my original error report because I was only testing in FP16 GPU Therefore, I've gone back to your previous version and tested as above. I get the same results.

In summary, only FP16 GPU fails without bounding boxes.

Thanks again.

TNTWEN commented 3 years ago

@at1993 In my test, the master branch+ 2021.1 yolov3demo +2021.1 Model Optimizer+2021.1 inference engine+ CPU will crash the program.So i fix it in OpenVINO2021.1 branch.

i have tested full yolov4 GPU FP16 with this version's code https://github.com/TNTWEN/OpenVINO-YOLOV4/tree/OpenVINO2021.1:

C:\Users\tianwen\Desktop\OpenVINO-YOLOV4-master>python "C:\Program Files (x86)\Intel\openvino_2021\deployment_tools\model_optimizer\mo.py" --input_model frozen_darknet_yolov4_model.pb --transformations_config yolov4.json --batch 1 --reverse_input_channels --data_type FP16
Model Optimizer arguments:
Common parameters:
        - Path to the Input Model:      C:\Users\tianwen\Desktop\OpenVINO-YOLOV4-master\frozen_darknet_yolov4_model.pb
        - Path for generated IR:        C:\Users\tianwen\Desktop\OpenVINO-YOLOV4-master\.
        - IR output name:       frozen_darknet_yolov4_model
        - Log level:    ERROR
        - Batch:        1
        - Input layers:         Not specified, inherited from the model
        - Output layers:        Not specified, inherited from the model
        - Input shapes:         Not specified, inherited from the model
        - Mean values:  Not specified
        - Scale values:         Not specified
        - Scale factor:         Not specified
        - Precision of IR:      FP16
        - Enable fusing:        True
        - Enable grouped convolutions fusing:   True
        - Move mean values to preprocess section:       None
        - Reverse input channels:       True
TensorFlow specific parameters:
        - Input model in text protobuf format:  False
        - Path to model dump for TensorBoard:   None
        - List of shared libraries with TensorFlow custom layers implementation:        None
        - Update the configuration file with input/output node names:   None
        - Use configuration file used to generate the model with Object Detection API:  None
        - Use the config file:  None
Model Optimizer version:        2021.1.0-1237-bece22ac675-releases/2021/1

[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: C:\Users\tianwen\Desktop\OpenVINO-YOLOV4-master\.\frozen_darknet_yolov4_model.xml
[ SUCCESS ] BIN file: C:\Users\tianwen\Desktop\OpenVINO-YOLOV4-master\.\frozen_darknet_yolov4_model.bin
[ SUCCESS ] Total execution time: 83.99 seconds.

C:\Users\tianwen\Desktop\OpenVINO-YOLOV4-master>python 2021_object_detection_demo_yolov3_async.py -i cam -d GPU -m C:\Users\tianwen\Desktop\OpenVINO-YOLOV4-master\frozen_darknet_yolov4_model.xml
[ INFO ] Creating Inference Engine...
[ INFO ] Loading network
[ INFO ] Preparing inputs
[ INFO ] Loading model to the plugin
[ INFO ] Starting inference...
To close the application, press 'CTRL+C' here or switch to the output window and press ESC key
To switch between min_latency/user_specified modes, press TAB key in the output window
2021.py:228: DeprecationWarning: 'layers' property of IENetwork class is deprecated. For iteration over network please use get_ops()/get_ordered_ops() methods from nGraph Python API
  out_blob = out_blob.buffer.reshape(net.layers[net.layers[layer_name].parents[0]].out_data[0].shape)

Mode: USER_SPECIFIED
Latency: 748.3 ms
FPS: 1.3

The bounding boxes are also correct.

These questions are really confusing，it seems to be related to the device or driver.Hope that more people can participate in the test in the future

TNTWEN commented 3 years ago

https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_windows.html#Install-GPU

Now OpenVINO's GPU installation method has also changed

at1993 commented 3 years ago

My GPU driver is OK, although it is a lower model than yours . gpu

TNTWEN commented 3 years ago

@at1993 Have you tried my OpenVINO2021.1 branch?

at1993 commented 3 years ago

Yes I have. I've also tried it on a different machine with a different Intel GPU, with the same problem.

gpu

at1993 commented 3 years ago

I thought I had. I will verify and report back.

TNTWEN commented 3 years ago

This is my Intel GPU. It's really confusing~

at1993 commented 3 years ago

Sorry, I must have downloaded the master branch this morning. I am downloading the OpenVINO2021.1 branch now. Will report back.

at1993 commented 3 years ago

Using the branch from OpenVINO2020.1 but regret to say that I am getting the same problem.

TNTWEN commented 3 years ago

Could you provide the detail commands and the result you get?

at1993 commented 3 years ago

Will repeat my steps now. Stay tuned.

at1993 commented 3 years ago

Hi TNTWEN, below are the steps:

D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1>python convert_weights_pb.py --class_names cfg/coco.names --weights_file yolov4.weights --data_format NHWC WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons
https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From convert_weights_pb.py:52: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

WARNING:tensorflow:From convert_weights_pb.py:38: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W1030 12:31:57.095026 21836 module_wrapper.py:139] From convert_weights_pb.py:38: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From convert_weights_pb.py:40: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W1030 12:31:57.110657 21836 module_wrapper.py:139] From convert_weights_pb.py:40: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From C:\Users\MeAtPlay\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use layer.__call__ method instead. W1030 12:31:57.126287 21836 deprecation.py:323] From C:\Users\MeAtPlay\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use layer.__call__ method instead. WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\yolo_v4.py:110: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

W1030 12:31:59.019663 21836 module_wrapper.py:139] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\yolo_v4.py:110: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

WARNING:tensorflow:From convert_weights_pb.py:42: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

W1030 12:31:59.797862 21836 module_wrapper.py:139] From convert_weights_pb.py:42: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:93: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

W1030 12:32:03.285157 21836 module_wrapper.py:139] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:93: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

WARNING:tensorflow:From convert_weights_pb.py:47: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

W1030 12:32:05.092739 21836 module_wrapper.py:139] From convert_weights_pb.py:47: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:52: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

W1030 12:32:09.646545 21836 module_wrapper.py:139] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:52: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:53: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.convert_variables_to_constants W1030 12:32:10.068444 21836 deprecation.py:323] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:53: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.convert_variables_to_constants WARNING:tensorflow:From C:\Users\MeAtPlay\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.extract_sub_graph W1030 12:32:10.068444 21836 deprecation.py:323] From C:\Users\MeAtPlay\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.extract_sub_graph INFO:tensorflow:Froze 541 variables. I1030 12:32:10.875267 21836 graph_util_impl.py:334] Froze 541 variables. INFO:tensorflow:Converted 541 variables to const ops. I1030 12:32:11.682537 21836 graph_util_impl.py:394] Converted 541 variables to const ops. WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:56: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

W1030 12:32:11.753525 21836 module_wrapper.py:139] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:56: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

1808 ops written to frozen_darknet_yolov4_model.pb.

at1993 commented 3 years ago

D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1>"C:\Program Files (x86)\IntelSWTools\openvino_2021\bin\setupvars.bat" Python 3.6.5 [setupvars.bat] OpenVINO environment initialized

at1993 commented 3 years ago

D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1>python "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo.py" --input_model frozen_darknet_yolov4_model.pb --transformations_config yolov4.json --batch 1 --reverse_input_channels --data_type FP16 Model Optimizer arguments: Common parameters:

Path to the Input Model: D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\frozen_darknet_yolov4_model.pb
Path for generated IR: D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1.
IR output name: frozen_darknet_yolov4_model
Log level: ERROR
Batch: 1
Input layers: Not specified, inherited from the model
Output layers: Not specified, inherited from the model
Input shapes: Not specified, inherited from the model
Mean values: Not specified
Scale values: Not specified
Scale factor: Not specified
Precision of IR: FP16
Enable fusing: True
Enable grouped convolutions fusing: True
Move mean values to preprocess section: None
Reverse input channels: True TensorFlow specific parameters:
Input model in text protobuf format: False
Path to model dump for TensorBoard: None
List of shared libraries with TensorFlow custom layers implementation: None
Update the configuration file with input/output node names: None
Use configuration file used to generate the model with Object Detection API: None
Use the config file: None Model Optimizer version: 2021.1.0-1237-bece22ac675-releases/2021/1

[ SUCCESS ] Generated IR version 10 model. [ SUCCESS ] XML file: D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1.\frozen_darknet_yolov4_model.xml [ SUCCESS ] BIN file: D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1.\frozen_darknet_yolov4_model.bin [ SUCCESS ] Total execution time: 50.41 seconds.

D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1>

at1993 commented 3 years ago

Running demo program object_detection_demo_yolov3_async using CPU

object_detection_demo_yolov3_async -i C:\DVCell\Common64B\Media\WaikikiBeach.mp4 -m C:\DVCell\Common64\models\frozen_darknet_yolov4_model.xml -d CPU InferenceEngine: API version ......... 2.1 Build ........... 2021.1.0-1237-bece22ac675-releases/2021/1 [ INFO ] Parsing input parameters [ INFO ] Loading Inference Engine [ INFO ] Device info: CPU MKLDNNPlugin version ......... 2.1 Build ........... 2021.1.0-1237-bece22ac675-releases/2021/1 [ INFO ] Loading network files [ INFO ] Checking that the inputs are as the demo expects [ INFO ] Checking that the outputs are as the demo expects [ INFO ] Loading model to the device MFX: Unsupported extension: C:\DVCell\Common64B\Media\WaikikiBeach.mp4 To close the application, press 'CTRL+C' here or switch to the output window and press ESC key To switch between sync/async modes, press TAB key in the output window ^C

TNTWEN commented 3 years ago

I found i upload a wrong file in branch OpenVINO2021.1 . i have fixed it .You could try again now. And i will verify your commands

at1993 commented 3 years ago

OK. I'll try it now.

at1993 commented 3 years ago

I've tried the latest. No difference though. Below are the commands:

python convert_weights_pb.py --class_names cfg/coco.names --weights_file yolov4.weights --data_format NHWC WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons
https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From convert_weights_pb.py:52: The name tf.app.run is deprecated. Please use tf.compat.v1.app.run instead.

WARNING:tensorflow:From convert_weights_pb.py:38: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

W1030 12:49:03.366822 4880 module_wrapper.py:139] From convert_weights_pb.py:38: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From convert_weights_pb.py:40: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

W1030 12:49:03.366822 4880 module_wrapper.py:139] From convert_weights_pb.py:40: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From C:\Users\MeAtPlay\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use layer.__call__ method instead. W1030 12:49:03.373329 4880 deprecation.py:323] From C:\Users\MeAtPlay\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use layer.__call__ method instead. WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\yolo_v4.py:128: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

W1030 12:49:05.227727 4880 module_wrapper.py:139] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\yolo_v4.py:128: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.

WARNING:tensorflow:From convert_weights_pb.py:42: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

W1030 12:49:05.977709 4880 module_wrapper.py:139] From convert_weights_pb.py:42: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:93: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

W1030 12:49:06.078722 4880 module_wrapper.py:139] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:93: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.

WARNING:tensorflow:From convert_weights_pb.py:47: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

W1030 12:49:07.886029 4880 module_wrapper.py:139] From convert_weights_pb.py:47: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:52: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

W1030 12:49:12.331326 4880 module_wrapper.py:139] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:52: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:53: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.convert_variables_to_constants W1030 12:49:12.736792 4880 deprecation.py:323] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:53: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.convert_variables_to_constants WARNING:tensorflow:From C:\Users\MeAtPlay\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.extract_sub_graph W1030 12:49:12.736792 4880 deprecation.py:323] From C:\Users\MeAtPlay\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use tf.compat.v1.graph_util.extract_sub_graph INFO:tensorflow:Froze 541 variables. I1030 12:49:13.498397 4880 graph_util_impl.py:334] Froze 541 variables. INFO:tensorflow:Converted 541 variables to const ops. I1030 12:49:14.320250 4880 graph_util_impl.py:394] Converted 541 variables to const ops. WARNING:tensorflow:From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:56: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

W1030 12:49:14.380692 4880 module_wrapper.py:139] From D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\utils.py:56: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.

1808 ops written to frozen_darknet_yolov4_model.pb.

at1993 commented 3 years ago

"C:\Program Files (x86)\IntelSWTools\openvino_2021\bin\setupvars.bat" Python 3.6.5 [setupvars.bat] OpenVINO environment initialized

at1993 commented 3 years ago

python "C:\Program Files (x86)\IntelSWTools\openvino_2021\deployment_tools\model_optimizer\mo.py" --input_model frozen_darknet_yolov4_model.pb --transformations_config yolov4.json --batch 1 --reverse_input_channels --data_type FP16 Model Optimizer arguments: Common parameters:

Path to the Input Model: D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1\frozen_darknet_yolov4_model.pb
Path for generated IR: D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1.
IR output name: frozen_darknet_yolov4_model
Log level: ERROR
Batch: 1
Input layers: Not specified, inherited from the model
Output layers: Not specified, inherited from the model
Input shapes: Not specified, inherited from the model
Mean values: Not specified
Scale values: Not specified
Scale factor: Not specified
Precision of IR: FP16
Enable fusing: True
Enable grouped convolutions fusing: True
Move mean values to preprocess section: None
Reverse input channels: True TensorFlow specific parameters:
Input model in text protobuf format: False
Path to model dump for TensorBoard: None
List of shared libraries with TensorFlow custom layers implementation: None
Update the configuration file with input/output node names: None
Use configuration file used to generate the model with Object Detection API: None
Use the config file: None Model Optimizer version: 2021.1.0-1237-bece22ac675-releases/2021/1

[ SUCCESS ] Generated IR version 10 model. [ SUCCESS ] XML file: D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1.\frozen_darknet_yolov4_model.xml [ SUCCESS ] BIN file: D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1.\frozen_darknet_yolov4_model.bin [ SUCCESS ] Total execution time: 51.83 seconds.

at1993 commented 3 years ago

Below are the output from Intel C++ demo program

object_detection_demo_yolov3_async -i C:\DVCell\Common64B\Media\WaikikiBeach.mp4 -m C:\DVCell\Common64\models\frozen_darknet_yolov4_model.xml -d CPU InferenceEngine: API version ......... 2.1 Build ........... 2021.1.0-1237-bece22ac675-releases/2021/1 [ INFO ] Parsing input parameters [ INFO ] Loading Inference Engine [ INFO ] Device info: CPU MKLDNNPlugin version ......... 2.1 Build ........... 2021.1.0-1237-bece22ac675-releases/2021/1 [ INFO ] Loading network files [ INFO ] Checking that the inputs are as the demo expects [ INFO ] Checking that the outputs are as the demo expects [ INFO ] Loading model to the device MFX: Unsupported extension: C:\DVCell\Common64B\Media\WaikikiBeach.mp4 To close the application, press 'CTRL+C' here or switch to the output window and press ESC key To switch between sync/async modes, press TAB key in the output window ^C cpu

at1993 commented 3 years ago

object_detection_demo_yolov3_async -i C:\DVCell\Common64B\Media\WaikikiBeach.mp4 -m C:\DVCell\Common64\models\frozen_darknet_yolov4_model.xml -d GPU InferenceEngine: API version ......... 2.1 Build ........... 2021.1.0-1237-bece22ac675-releases/2021/1 [ INFO ] Parsing input parameters [ INFO ] Loading Inference Engine [ INFO ] Device info: GPU clDNNPlugin version ......... 2.1 Build ........... 2021.1.0-1237-bece22ac675-releases/2021/1 [ INFO ] Loading network files [ INFO ] Checking that the inputs are as the demo expects [ INFO ] Checking that the outputs are as the demo expects [ INFO ] Loading model to the device MFX: Unsupported extension: C:\DVCell\Common64B\Media\WaikikiBeach.mp4 To close the application, press 'CTRL+C' here or switch to the output window and press ESC key To switch between sync/async modes, press TAB key in the output window ^C gpu

There are no bounding boxes.

TNTWEN commented 3 years ago

try this https://github.com/TNTWEN/OpenVINO-YOLOV4/blob/OpenVINO2021.1/2021_object_detection_demo_yolov3_async.py python demo

cd D:\Programming\CPP\OpenVINO-YOLOV4-OpenVINO2021.1
python 2021_object_detection_demo_yolov3_async.py -i C:\DVCell\Common64B\Media\WaikikiBeach.mp4 -m C:\DVCell\Common64\models\frozen_darknet_yolov4_model.xml  -d GPU

at1993 commented 3 years ago

OK, I've tried the python demo. Same problem.

gpu

at1993 commented 3 years ago

@TNTWEN I am just wondering if you can upload your convert IR files (frozen_darknet_yolov4_model.bin/xml) so that I can test it with my hardware. Note: only the FP16 version.

TNTWEN commented 3 years ago

@at1993 Maybe I know what causes such a problem.This is related to the version of tensorflow OpenVINO2021.1 requires tensorflow1.15.4 tf1.15.4+2021.1 Model Optimizer+FP16+2021.1+GPU inference engine .I reproduced your problem I have been using my previous .pb model. So master branch +FP32+CPU meet error. But master/OpenVINO2021.1 branch +FP16+GPU is OK When i use tf1.15.4 .pb model ,CPU works well and GPU meets problem like yours

It seems that OpenVINO2021.1 is Optimized for tf1.15.4. yolov4 CPU runs faster .But it also causes GPU+fp16 detect no bounding boxes.

TNTWEN commented 3 years ago

And OpenVINO2021.1 inference engine makes tf 1.12.0 slower!

TNTWEN commented 3 years ago

I located the problem. it's the mish activation function activation_fn=lambda x:x* tf.math.tanh(tf.math.softplus(x)) OpenVINO 2021.1 doesn't seem to work well with mish activation function in GPU+FP16 But CPU+FP16 works well. I think this is the bug of openvino when GPU infers fp16

TNTWEN commented 3 years ago

@at1993 https://drive.google.com/file/d/1VnFTF54DlaD1S3O50zR4yonzl6XYpp2R/view?usp=sharing This is previous tf(i forgot the exact version)+2021.1 Model Optimizer +FP16 CPU will crash the program GPU is OK （but it takes a very long time to load the model)

But FPS is very slow

I tried to remove the mish activation function, use tf1.15.4+OpenVINO2021.1 ,bounding box is normal. so it can be inferred that openvino optimizes the mish activation function in the 2021.1 to make it faster.But GPU doesn't seem to handle fp16 mish properly

at1993 commented 3 years ago

I located the problem. it's the mish activation function activation_fn=lambda x:x* tf.math.tanh(tf.math.softplus(x)) OpenVINO 2021.1 doesn't seem to work well with mish activation function in GPU+FP16 But CPU+FP16 works well. I think this is the bug of openvino when GPU infers fp16

Thank you for your effort. One question though, Why does it work for YoloV4 tiny model in FP16 GPU?

TNTWEN commented 3 years ago

@at1993 Because only full yolov4 uses mish. yolov4tiny only uses leaky-relu

at1993 commented 3 years ago

OK, that makes sense. Thanks.

at1993 commented 3 years ago

OpenVINO 2021.2 was released yesterday. Unfortunately, this load still does not fix this problem :(

KennyDizi commented 3 years ago

interested in

TNTWEN commented 3 years ago

Hi! @at1993 https://github.com/TNTWEN/OpenVINO-YOLOV4/issues/30 Could you help verify if OpenVINO2021.2.185 fixed this bug? Thanks a lot !

at1993 commented 3 years ago

Yes, I did tried Openvino2021.2.185. It did not work. Not only that, their sample python/c++ demo code would not work yolov4 either. More info here. https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/Converting-Public-Model-yolo-v4-tf-to-IR/m-p/1237698

TNTWEN commented 3 years ago

@at1993 It seems that Openvino2021.2.185 removes python inference demo .So i add it here https://github.com/TNTWEN/OpenVINO-YOLOV4/tree/master/pythondemo

C++ inference demo still works .For windows,it's here:C:\Program Files (x86)\Intel\openvino_2021.2.185\inference_engine\demos\multi_channel\object_detection_demo_yolov3

In my test ,the mish GPU bug also still exists. But now yolov4-tiny-3l is a very good chioce:https://github.com/TNTWEN/OpenVINO-YOLOV4/tree/v4-tiny-3l

^-^

TNTWEN commented 3 years ago

@at1993
OpenVINO 2021.3 has been released. I have tested YOLOv4+gpu+fp16. It seems that the bug is fixed Would you also help me verify it? Thanks a lot!!

at1993 commented 3 years ago

@TNTWEN thanks for letting me know. I've tested it using the following:

YOLOv4 model + FP16 + GPU

Python script object_detection_demo_yolov3_async.py result is good with FPS: 6

C++ object_detection_demo.exe -d GPU result is good with FPS: 9.8

Custom C++ library using OpenVino 2021.3 using GPU result is good with FPS: 9.5

NOTE: I was hoping to increase the inference frame rate by changing the network size from 608 to 320 but it does not seems to make a difference? Any ideas?

Thanks.

TNTWEN commented 3 years ago

@at1993 Under normal circumstances, the smaller the image size is, the faster it will be. I think you could check your IRmodel using netron like this: check if the input image size is correct.

Recently,i met a single class object detection task.I use yolov4 and https://github.com/TNTWEN/Pruned-OpenVINO-YOLO to prune this model. Finally the size of .weights file is compressed from 245Mb to 800Kb with little loss of precision. And CPU(I5 8250U）+Pruned yolov4 + C++ inference demo reached 13FPS.

So i think the best way to speed up the model at this stage is to prune the model,remove model redundancy to the maximum extent.But you need to have enough GPU resources to prune and train the model!

at1993 commented 3 years ago

Thank you, I will try to prune it using your tool.

TNTWEN / OpenVINO-YOLOV4

YoloV4 broken with OpenVINO 2021.1 #18