Mxnet's yolov3 (gluoncv) w/ darknet weights not being optimized.

[ZiyueWangUoB]

Trying to use gluoncv's model zoo models in model optimizer, but always error (question 38). The .params and .json files of the model are in the correct format respectively, but always says one layer is not in the correct shape? I read here https://github.com/openvinotoolkit/openvino/issues/488 that it is supported, so not sure where the issue is. I had my own trained weights with the default exported json file straight out using gluoncv, but always question 38.

UPDATE: This has been resolved, scroll down for latest error.

[ZiyueWangUoB]

Update: I've found the source of the problem. MO doesn't like the first layer in yolo3's symbol file. The input for the layer is [[0,0,0],[1,0,0]] and for some reason the first element causes the shape to become negative (initial input shape of [1,3,608,608] becomes [1,3,608,-1]) and the code is unable to continue. Removing this however, makes MO think the rest of the layers in the file are useless and they are deleted. The error question is now #40. Any help will be appreciated.

UPDATE: This has been resolved, scroll down.

[ZiyueWangUoB]

For more information, here is the error log

[ 2020-08-14 12:26:48,063 ] [ DEBUG ] [ infer:130 ]  Partial infer for _defaultpreprocess0_broadcast_minus0
[ 2020-08-14 12:26:48,063 ] [ DEBUG ] [ infer:131 ]  Op: Sub
[ 2020-08-14 12:26:48,063 ] [ DEBUG ] [ infer:132 ]  Inputs:
[ 2020-08-14 12:26:48,063 ] [ DEBUG ] [ infer:32 ]  input[0]: shape = [  1   3 608 608], value = <UNKNOWN>
[ 2020-08-14 12:26:48,064 ] [ DEBUG ] [ infer:32 ]  input[1]: shape = [1 1 1 3], value = [[[[123.675 116.28  103.53 ]]]]
[ 2020-08-14 12:26:48,064 ] [ DEBUG ] [ infer:145 ]  Outputs:
[ 2020-08-14 12:26:48,064 ] [ DEBUG ] [ infer:32 ]  output[0]: shape = [  1   3 608  -1], value = <UNKNOWN>
[ ERROR ]  Shape [  1   3 608  -1] is not fully defined for output 0 of "_defaultpreprocess0_broadcast_minus0". Use --input_shape with positive integers to override model input shapes.
[ ERROR ]  Cannot infer shapes or values for node "_defaultpreprocess0_broadcast_minus0".
[ ERROR ]  Not all output shapes were inferred or fully defined for node "_defaultpreprocess0_broadcast_minus0". 
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #40. 
[ ERROR ]  

Json file has been converted to txt for readability.

yolo3_darknet53_voc-symbol.txt

UPDATE: This part has been fixed, the input shape was Channels last so I had to input shape [1,608,608,3]. However the following problems still exist.

[ZiyueWangUoB]

Update2: Saw the instructions for gluoncv models on this page, tried running for yolo3 mobilenet and another bug appeared during conversion

[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]  Exception occurred during running replacer "None (<class 'extensions.front.mxnet.ssd_detection_output_replacer.SsdPatternDetectionOutputReplacer'>)": 
[ ERROR ]  Traceback (most recent call last):
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 288, in apply_transform
    for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern)
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/middle/pattern_match.py", line 58, in for_graph_and_each_sub_graph_recursively
    func(graph)
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/front/common/replacement.py", line 148, in find_and_replace_pattern
    apply_pattern(graph, action=self.replace_sub_graph, **self.pattern())
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/middle/pattern_match.py", line 95, in apply_pattern
    action(graph, match)
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/extensions/front/mxnet/ssd_detection_output_replacer.py", line 129, in replace_sub_graph
    assert concat_names[0] in ssd_concats
AssertionError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/main.py", line 309, in main
    ret_code = driver(argv)
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/main.py", line 270, in driver
    ret_res = emit_ir(prepare_ir(argv), argv)
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/main.py", line 235, in prepare_ir
    graph = unified_pipeline(argv)
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/pipeline/unified.py", line 29, in unified_pipeline
    class_registration.ClassType.BACK_REPLACER
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 334, in apply_replacements
    apply_replacements_list(graph, replacers_order)
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 324, in apply_replacements_list
    num_transforms=len(replacers_order))
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/logger.py", line 124, in wrapper
    function(*args, **kwargs)
  File "/opt/intel/openvino_2020.4.287/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 312, in apply_transform
    )) from err
Exception: Exception occurred during running replacer "None (<class 'extensions.front.mxnet.ssd_detection_output_replacer.SsdPatternDetectionOutputReplacer'>)": 

[ ERROR ]  ---------------- END OF BUG REPORT --------------
[ ERROR ]  -------------------------------------------------

I followed the instructions step by step for yolo3_mobilenet1.0_voc. This happens when running with the --enable_ssd_gluoncv command. If that is removed, however, the error shows "_contrib_box_nms" not not supported. Which doesn't make sense as yolo is supposed to be supported?

[ZiyueWangUoB]

Last update I promise: Ok following the documentation with the incorrect transformation config location (extra mo), I've got it working with the pretrained models (and my own trained model). Be ware, you have to use the --transformations_config file or else it DOESN'T WORK. Be sure to have the same input shapes too.