openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
https://docs.openvino.ai
Apache License 2.0
6.78k stars 2.16k forks source link

converting deepspeech german language model #3619

Closed OrysB closed 3 years ago

OrysB commented 3 years ago
System information (version)
Detailed description

I tried to convert a german language deepspeech tensorflow model and got the following error:

[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.load.tf.loader.TFLoader'>): Unexpected exception happened during extracting attributes for node metadata_alphabet. Original exception message: 'ascii' codec can't decode byte 0xc3 in position 6: ordinal not in range(128)

Steps to reproduce

tensorflow model optimizer with the following parameters:

./mo_tf.py --input_model=./models/output_graph.pb --input="input_node,previous_state_h,previous_state_c/read" --input_shape=[1,16,19,26],[1,2048],[1,2048] --output=logits,cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd,cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd_1 --freeze_placeholder_with_value="input_lengths->[16]" --disable_nhwc_to_nchw

The Problems are probably due to the german Umlaute, i.e. ö, ü, ä, ect. What do I do now?

OrysB commented 3 years ago

The mo_tf.py loader <class 'extensions.load.tf.loader.TFLoader'> extracts the node metadata.alphabet and its attributes using an ascii decoder. 0cx3 is the first half of utf-8 hex code in some UTF-8 characters. Hence the Error. The function responsible for extracting attributes is that is "extract_node_attrs" found in mo.front.extractor, where we find the error: 'Unexpected exception happened during extracting attributes for node {}.' , which looks familiar.

Running the command with --log_level=DEBUG:

[ WARNING ] Node attributes: {'pb': name: "metadata_alphabet" op: "Const" attr { key: "dtype" value { type: DT_STRING } } attr { key: "value" value { tensor { dtype: DT_STRING tensor_shape { dim { size: 1 } } string_val: "\037\000\036\000\002\000\303\237\035\000\002\000\303\274\034\000\002\000\303\266\033\000\002\000\303\244\032\000\001\000z\031\000\001\000y\030\000\001\000x\027\000\001\000w\026\000\001\000v\025\000\001\000u\024\000\001\000t\023\000\001\000s\022\000\001\000r\021\000\001\000q\006\000\001\000f\005\000\001\000e\004\000\001\000d\003\000\001\000c\000\000\001\000 \001\000\001\000a\002\000\001\000b\007\000\001\000g\010\000\001\000h\t\000\001\000i\n\000\001\000j\013\000\001\000k\014\000\001\000l\r\000\001\000m\016\000\001\000n\017\000\001\000o\020\000\001\000p" } } } , '_in_ports': {}, '_out_ports': {}, 'kind': 'op', 'name': 'metadata_alphabet', 'op': 'Const'} [ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.load.tf.loader.TFLoader'>): Unexpected exception happened during extracting attributes for node metadata_alphabet. Original exception message: 'ascii' codec can't decode byte 0xc3 in position 6: ordinal not in range(128) [ 2020-12-15 21:15:41,307 ] [ DEBUG ] [ main:310 ] Traceback (most recent call last): File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/front/extractor.py", line 421, in extract_node_attrs supported, new_attrs = extractor(Node(graph, node)) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/extensions/load/tf/loader.py", line 102, in extract_node_attrs(graph, lambda node: tf_op_extractor(node, check_for_duplicates(tf_op_extractors))) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/front/tf/extractor.py", line 89, in tf_op_extractor attrs = tf_op_extractorsop File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/front/common/register_custom_ops.py", line 96, in node, cls, disable_omitting_optional, enable_flattening_optional_params), File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/front/common/register_custom_ops.py", line 29, in extension_extractor supported = ex.extract(node) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/extensions/front/tf/const_ext.py", line 32, in extract 'value': tf_tensor_content(pb_tensor.dtype, shape, pb_tensor), File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/front/tf/extractors/utils.py", line 75, in tf_tensor_content value = np.array(type_helper1, dtype=type_helper[0]) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 6: ordinal not in range(128)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/utils/class_registration.py", line 288, in apply_transform for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/middle/pattern_match.py", line 58, in for_graph_and_each_sub_graph_recursively func(graph) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/extensions/load/loader.py", line 27, in find_and_replace_pattern self.load(graph) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/extensions/load/tf/loader.py", line 102, in load extract_node_attrs(graph, lambda node: tf_op_extractor(node, check_for_duplicates(tf_op_extractors))) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/front/extractor.py", line 429, in extract_node_attrs ) from e mo.utils.error.Error: Unexpected exception happened during extracting attributes for node metadata_alphabet. Original exception message: 'ascii' codec can't decode byte 0xc3 in position 6: ordinal not in range(128)

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/main.py", line 297, in main ret_code = driver(argv) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/main.py", line 264, in driver ret_res = emit_ir(prepare_ir(argv), argv) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/main.py", line 233, in prepare_ir graph = unified_pipeline(argv) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/pipeline/unified.py", line 29, in unified_pipeline class_registration.ClassType.BACK_REPLACER File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/utils/class_registration.py", line 338, in apply_replacements apply_replacements_list(graph, replacers_order) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/utils/class_registration.py", line 328, in apply_replacements_list num_transforms=len(replacers_order)) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/utils/logger.py", line 124, in wrapper function(*args, **kwargs) File "/home/hamlet/Documents/Informatik/Projekte/Computervision/packages/openvino/model-optimizer/mo/utils/class_registration.py", line 308, in apply_transform )) from err mo.utils.error.Error: Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.load.tf.loader.TFLoader'>): Unexpected exception happened during extracting attributes for node metadata_alphabet. Original exception message: 'ascii' codec can't decode byte 0xc3 in position 6: ordinal not in range(128)

OrysB commented 3 years ago

whoops closed by mistake ^^

jgespino commented 3 years ago

Hi @OrysB

I am seeing the same error on Ubuntu 18.04 with the new OpenVINO toolkit 2021.2 release. Let me check with my peers and find out if this model is supported.

Regards, Jesus

OrysB commented 3 years ago

Hey @jgespino , Thank you very much! I don't have a lot of experience, so I feel like running against a wall trying to solve this. ^^

jgespino commented 3 years ago

Hi @OrysB

No problem, we have opened a bug with the dev team to take a look. I will let you know once I know more.

Regards, Jesus

Ref. 45435

jgespino commented 3 years ago

Hi @OrysB

Dev team has created a pull request with a potential fix, once it's merged please try building OpenVINO toolkit from source.

Also, the MO command should specify --input="input_node,previous_state_h,previous_state_c" instead of --input="input_node,previous_state_h,previous_state_c/read".

Regards, Jesus

OrysB commented 3 years ago

Hey @jgespino YAY!!! 🎉

python ./mo_tf.py --input_model=./models/output_graph.pb --input="input_node,previous_state_h,previous_state_c" --input_shape=[1,16,19,26],[1,2048],[1,2048] --output=logits,cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd,cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd_1 --freeze_placeholder_with_value="input_lengths->[16]" --disable_nhwc_to_nchw Model Optimizer arguments: Common parameters:

[ SUCCESS ] Generated IR version 10 model. [ SUCCESS ] XML file: [path_to]/DeepSpeech.OpenVINO/./output_graph.xml [ SUCCESS ] BIN file: [path_to]/DeepSpeech.OpenVINO/./output_graph.bin [ SUCCESS ] Total execution time: 42.12 seconds. [ SUCCESS ] Memory consumed: 1146 MB.

Thank you very much for the help! I wish you and your peers a merry Christmas and happy new year!

Closing this issue.