Unable to convert deepspeech-0.6.1 model

[smitesh-sutaria]

Using https://github.com/FengYen-Chang/open_model_zoo/tree/master/demos/python_demos/deep_speech_demo able to convert deepspeech-0.5.1 model.

But when attempt it on 0.6.1 version, get the below error :

python3 ./mo_tf.py --input_model /root/deepspeech-0.6.0-models/output_graph.pb --freeze_placeholder_with_value "input_lengths->[16]" --input input_node,previous_state_h/read,previous_state_c/read --input_shape [1,16,19,26],[1,2048],[1,2048] --output Softmax,lstm_fused_cell/GatherNd,lstm_fused_cell/GatherNd_1 --disable_nhwc_to_nchw
Model Optimizer arguments:
Common parameters:
        - Path to the Input Model:      /root/deepspeech-0.6.0-models/output_graph.pb
        - Path for generated IR:        /opt/intel/openvino_2020.3.194/deployment_tools/model_optimizer/.
        - IR output name:       output_graph
        - Log level:    ERROR
        - Batch:        Not specified, inherited from the model
        - Input layers:         input_node,previous_state_h/read,previous_state_c/read
        - Output layers:        Softmax,lstm_fused_cell/GatherNd,lstm_fused_cell/GatherNd_1
        - Input shapes:         [1,16,19,26],[1,2048],[1,2048]
        - Mean values:  Not specified
        - Scale values:         Not specified
        - Scale factor:         Not specified
        - Precision of IR:      FP32
        - Enable fusing:        True
        - Enable grouped convolutions fusing:   True
        - Move mean values to preprocess section:       False
        - Reverse input channels:       False
TensorFlow specific parameters:
        - Input model in text protobuf format:  False
        - Path to model dump for TensorBoard:   None
        - List of shared libraries with TensorFlow custom layers implementation:        None
        - Update the configuration file with input/output node names:   None
        - Use configuration file used to generate the model with Object Detection API:  None
        - Use the config file:  None
Model Optimizer version:
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.front.user_data_repack.UserDataRepack'>): No node with name previous_state_h/read.
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #51.

[eaidova]

@smitesh-sutaria the error message "No node with name previous_state_h/read" speak that model does not contain layer with this name. This stale PR was not accepted to OMZ, why do you decide that it contains actual info? (especially, if take in account that at the moment of its creation, deepspeech0.6.1 has not been released at all :)

please try the following command for converting model:

python ./mo_tf.py --model_name "deepspeech_0.6.1" \
  --output_dir <output_ir_dir> \
  --input_model <path_to_tf_model>/output_graph.pb \
  --freeze_placeholder_with_value "input_lengths->[16]" \
  --input "input_node,previous_state_h,previous_state_c" \
  --input_shape "[1,16,19,26],[1,2048],[1,2048]" \
  --output "logits,cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd,cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/GatherNd_1" \
  --disable_nhwc_to_nchw \
  --log_level WARNING

P.S. I also need to warn you, if you want try demo from this PR on deepspeech0.6.1, it is also may be not suitable.

[smitesh-sutaria]

@eaidova Thank you for the inputs. The command you gave worked for converting the model but as you rightly pointed out, the demo might not work. Understand that v0.6.1 is not backward compatible with v0.5.1. The rationale behind usage of newer versions of deepspeech is to get better performance. Which version of deepspeech is officially supported ? What are the future roadmap in terms of supporting latest version of deepspeech ?

While using the 0.6.1 model with PR demo, below is the error i get : (mismatch with input , output layers)

python3 deepspeech_openvino_0.5.py -m /opt/intel/openvino_2020.3.194/deployment_tools/model_optimizer/output_graph.xml -i audio/2830-3980-0043.wav -a alphabet_b.txt [ INFO ] Creating Inference Engine [ INFO ] Loading network files: /opt/intel/openvino_2020.3.194/deployment_tools/model_optimizer/output_graph.xml /opt/intel/openvino_2020.3.194/deployment_tools/model_optimizer/output_graph.bin deepspeech_openvino_0.5.py:124: DeprecationWarning: Reading network using constructor is deprecated. Please, use IECore.read_network() method instead net = IENetwork(model=model_xml, weights=model_bin) [ INFO ] Preparing input blobs [ INFO ] Preparing output blobs [ INFO ] Loading model to the plugin Traceback (most recent call last): File "deepspeech_openvino_0.5.py", line 185, in sys.exit(main() or 0) File "deepspeech_openvino_0.5.py", line 174, in main 'input_node': [chunk]}) File "ie_api.pyx", line 478, in openvino.inference_engine.ie_api.ExecutableNetwork.infer File "ie_api.pyx", line 683, in openvino.inference_engine.ie_api.InferRequest.infer File "ie_api.pyx", line 685, in openvino.inference_engine.ie_api.InferRequest.infer File "ie_api.pyx", line 826, in openvino.inference_engine.ie_api.InferRequest._fill_inputs AssertionError: No input with name previous_state_c/read/placeholder_port_0 found in network

[smitesh-sutaria]

@FengYen-Chang : Your Inputs/Feedback please.

[eaidova]

@smitesh-sutaria, deepspeech model have some troubles with license, it stopped us to publish it as part OMZ, but we working on it... In the optimistic scenario, the updated demo can become a part of OMZ in the next release. I can say that deepspeech0.6.1 actually supported by openvino and successfully pass accuracy validation via accuracy checker, so the observed issue connected only with incompatibilities in John Feng's code. this error similar with what you sow during model conversion and connected with inputs renaming, please try to replace constants for input names with new:

• previous_state_c/read/placeholder_port_0 -> previous_state_c
• previous_state_h/read/placeholder_port_0 -> previous_state_h possibly you need to do the same for outputs.

[smitesh-sutaria]

@eaidova Thank you for the inputs. Will try it out and keep you posted. WER (word error rate) of the latest release 0.7.4 is 5.97%, while that of 0.6.1 is 7.5%. Any specific reason to go for 0.6.1 and not for the latest release 0.7.4? Are there plans/road-map to support 0.7.4 ?

[FengYen-Chang]

Thanks @eaidova. @smitesh-sutaria , really sorry for the late reply, I have updated the demo to version,0.6.1. Please check that repo. Thanks.

[FengYen-Chang]

For the version 0.7.4, you can use the same MO command which you convert the version 0.6.1 to convert the .pb file and then patch the parameter upper_frequency_limit_ which under function mfcc_mel_filiterbank_init from 4000 to 8000. After this patch, the demo should be run well.

[vladimir-dudnik]

@smitesh-sutaria do you have an additional questions on this?

[IRDonch]

This model has now been added to OMZ, so this should no longer be relevant. Closing.