search

TensorSpeech / TensorFlowASR

:zap: TensorFlowASR: Almost State-of-the-art Automatic Speech Recognition in Tensorflow 2. Supported languages that can use characters or subwords
https://huylenguyen.com/asr
Apache License 2.0
928 stars 243 forks source link

streaming_transducer can't convert h5 to tflite #112

Closed goreng2 closed 3 years ago

goreng2 commented 3 years ago

Hi,

I try to convert transducer .h5 to tflite using tflite_streaming_transducer.py. But It's not work...

First, I fix concrete_func = streaming_transducer.make_tflite_function().get_concrete_function() (remove greedy=True argument of make_tflite_function() because error, tflite_streaming_transducer.py's 60 line)

The below is the end of error result

 %42:3 = "tf.If"(%41, %40, %arg4, %arg2, %24) {else_branch = @"streaming_transducer_greedy/while/cond_else", is_stateless = true, then_branch = @"streaming_transducer_greedy/while/cond_then"} : (tensor<i1>, tensor<i32>, tensor<2x2x1x1024xf32>) -> (tensor<i32>, tensor<i32>, tensor<2x2x1x1024xf32>)
  %43 = "tfl.reshape"(%42#1, %cst_19) : (tensor<i32>, tensor<1xi32>) -> tensor<1xi32>
  %44 = "tf.TensorScatterUpdate"(%arg3, %1, %43) {device = ""} : (tensor<?xi32>, tensor<1x1xi32>, tensor<1xi32>) -> tensor<?xi32>
  %45 = "tfl.add"(%arg0, %cst_18) {fused_activation_function = "NONE"} : (tensor<i32>, tensor<i32>) -> tensor<i32>
  "std.return"(%45, %2, %42#0, %44, %42#2, %arg5, %arg6) : (tensor<i32>, tensor<i32>, tensor<i32>, tensor<?xi32>, tensor<2x2x1x1024xf32>, tensor<i32>, tensor<?x320xf32>) -> ()
}) {sym_name = "streaming_transducer_greedy/while_body", sym_visibility = "private", type = (tensor<i32>, tensor<i32>, tensor<i32>, tensor<?xi32>, tensor<2x2x1x1024xf32>, tensor<i32>, tensor<?x320xf32>) -> (tensor<i32>, tensor<i32>, tensor<i32>, tensor<?xi32>, tensor<2x2x1x1024xf32>, tensor<i32>, tensor<?x320xf32>)} : () -> ()

I give timestamp argument such as make_tflite_function(timestamp=False). But It's not work too

nglehuy commented 3 years ago

which tensorflow version are you using?

goreng2 commented 3 years ago

I'm using tensorflow=2.3 and make model at TensorflowASR=0.6.3

nglehuy commented 3 years ago

@JeongWoonPark try using tf-nightly to convert to tflite.

goreng2 commented 3 years ago

@usimarit Hi, I convert that with tf-nightly-cpu It work, Thanks.

goreng2 commented 3 years ago

Hi, @usimarit Thank you for your reply always.

I encounter another issue about tflite. Specifically, loading tflite model problem

  1. I make h5 model with tensorflow=2.3, TensorASR=0.6.3
  2. Can't convert h5 to tflite at the same env. So, using tf-nightly instead of tensorflow=2.3. It works. I get the tflite model
  3. After that, I test below code and encounter error 
    
import tensorflow as tf

tf.lite.Interpreter(model_path="./latest-0.6.3-20epoch.tflite")

at `tf-nightly`:

Traceback (most recent call last): File "", line 1, in File "path/lib/python3.8/site-packages/tensorflow/lite/python/interpreter.py", line 280, in init _interpreter_wrapper.CreateWrapperFromFile( ValueError: Did not get operators or tensors in subgraph 1.

at `tensorflow=2.3`:

Traceback (most recent call last): File "", line 1, in File "path\lib\site-packages\tensorflow\lite\python\interpreter.py", line 197, in init _interpreter_wrapper.CreateWrapperFromFile( ValueError: Op builtin_code out of range: 127. Are you using old TFLite binary with newer model?Registration failed.

nglehuy commented 3 years ago

@JeongWoonPark The issue is reproduced, I'm working on this.

nglehuy commented 3 years ago

@JeongWoonPark Please pull the newest code (aka v0.6.4) and try again :smile:

goreng2 commented 3 years ago

@usimarit Hi, It work! I success to convert and inference streaming_transducer model. In my case, The Model is made in TensorFlowASR=0.6.3 And is converted in TensorFlowASR=0.6.4. Both of them use Tensorflow-gpu=2.3. Thanks

And I have 2 questions. Q 1. I think concrete_func = streaming_transducer.make_tflite_function(greedy=True).get_concrete_function() should be modified to concrete_func = streaming_transducer.make_tflite_function().get_concrete_function() (remove greedy=True argument) at examples/streaming_transducer/tflite_streaming_transducer.py's 60 line. Is it right?

Q 2. At the inference, Segmentation fault occur when I test in tensorflow-gpu=2.3 In the tensorflow=2.3(CPU?), It work. tflite originally doesn't work at GPU?

nglehuy commented 3 years ago

@JeongWoonPark Q1: Okay I'll update the scripts :smile: Q2: From this doc the tflite does support gpu with some limited ops. Let me see if we can make the model support this :smile:

goreng2 commented 3 years ago

@usimarit Thank you! It's very helpful

goreng2 commented 3 years ago

Hi, @usimarit

I try to convert streaming_transducer .h5 to .tflite

tfasr src version that I tried: 0.6.4, 0.7.2, 0.7.5 tensorflow version: 2.3.0, 2.4.1 trained model version: 0.6.3

It works only at tfasr 0.6.4 with tensorflow 2.3.0

when I use tensorflow 2.4.1 at all(upper) tfasr version. The below error occurs (error msg is so long)

Traceback (most recent call last):
  File "tflite_streaming_transducer.py", line 66, in <module>
    tflite_model = converter.convert()
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow/lite/python/lite.py", line 1117, in convert
    return super(TFLiteConverterV2, self).convert()
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow/lite/python/lite.py", line 920, in convert
    _convert_to_constants.convert_variables_to_constants_v2_as_graph(
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow/python/framework/convert_to_constants.py", line 1110, in convert_variables_to_constants_v2_as_graph
    frozen_func = _construct_concrete_function(func, output_graph_def,
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow/python/framework/convert_to_constants.py", line 1001, in _construct_concrete_function
    new_func = wrap_function.function_from_graph_def(output_graph_def,
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow/python/eager/wrap_function.py", line 650, in function_from_graph_def
    wrapped_import = wrap_function(_imports_graph_def, [])
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow/python/eager/wrap_function.py", line 620, in wrap_function
    return WrappedFunction(
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow/python/eager/wrap_function.py", line 229, in __init__
    context.context().add_function_def(f)
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow/python/eager/context.py", line 1137, in add_function_def
    pywrap_tfe.TFE_ContextAddFunctionDef(self._handle, fdef_string,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Attempting to add a duplicate function with name: __inference_standard_lstm_13331 where the previous and current definitions differ. Previous definiton: signature {
  name: "__inference_standard_lstm_13331"
  input_arg {
    name: "inputs"
    type: DT_FLOAT
  }
...

I think that It seems a problem of tensorflow version. But when I use tensorflow 2.3.0 with tfasr 0.7.2, tfasr 0.7.5, The below error occurs...

Traceback (most recent call last):
  File "tflite_streaming_transducer.py", line 23, in <module>
    from tensorflow_asr.featurizers.speech_featurizers import TFSpeechFeaturizer
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_asr/featurizers/speech_featurizers.py", line 22, in <module>
    import tensorflow_io as tfio
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_io/__init__.py", line 17, in <module>
    from tensorflow_io.core.python.api.v0 import *  # pylint: disable=wildcard-import
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_io/core/python/api/v0/__init__.py", line 18, in <module>
    from tensorflow_io.core.python.ops.io_dataset import IODataset
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_io/core/python/ops/__init__.py", line 95, in <module>
    plugin_ops = _load_library("libtensorflow_io_plugins.so", "fs")
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_io/core/python/ops/__init__.py", line 63, in _load_library
    l = load_fn(f)
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_io/core/python/ops/__init__.py", line 55, in <lambda>
    load_fn = lambda f: tf.experimental.register_filesystem_plugin(f) is None
AttributeError: module 'tensorflow._api.v2.experimental' has no attribute 'register_filesystem_plugin'

In my case, Only tfasr 0.6.4 with tensorflow 2.3.0 can convert streaming_transducer .h5 to .tflite

How to convert tflite at latest tfasr version?

Thanks

nglehuy commented 3 years ago

@JeongWoonPark I remember tflite is not working with tf 2.4.x, could you try with tf-nightly? The message says that tensorflow_io import cause errror, I'll find a way to fix this.

nglehuy commented 3 years ago

@JeongWoonPark tensorflow-io 0.16.0 has bugs in tf 2.3.0, please install tf 2.3.2

goreng2 commented 3 years ago

Hi @usimarit In my case, Both tf 2.3.0 and tf 2.3.2 have same env: tensorflow-io 0.17.0. Therefore tf 2.3.2 also occurs same error related with tensorflow-io

I will inform you result after test tf-nightly

Thanks.

nglehuy commented 3 years ago

@JeongWoonPark tensorflow-io 0.17.0 is only for tf 2.4.x, please install tensorflow-io 0.16.0 for tf 2.3.x

goreng2 commented 3 years ago

Hi, @usimarit I try tensorflow-io 0.16.0 with tensorflow 2.3.2 at tfasr 0.7.5 And I got the below error

Traceback (most recent call last):
  File "/home/pjw1210/STT/streaming_transducer/convert_h5_to_tflite.py", line 5, in <module>
    from tensorflow_asr.models.streaming_transducer import StreamingTransducer
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_asr/models/streaming_transducer.py", line 19, in <module>
    from .transducer import Transducer
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_asr/models/transducer.py", line 22, in <module>
    from ..featurizers.text_featurizers import TextFeaturizer
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_asr/featurizers/text_featurizers.py", line 27, in <module>
    from . import wordpiece
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_asr/featurizers/wordpiece.py", line 34, in <module>
    import tensorflow_text as tft
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_text/__init__.py", line 21, in <module>
    from tensorflow_text.python import metrics
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_text/python/metrics/__init__.py", line 20, in <module>
    from tensorflow_text.python.metrics.text_similarity_metric_ops import *
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_text/python/metrics/text_similarity_metric_ops.py", line 28, in <module>
    gen_text_similarity_metric_ops = load_library.load_op_library(resource_loader.get_path_to_datafile('_text_similarity_metric_ops.so'))
  File "/home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow/python/framework/load_library.py", line 58, in load_op_library
    lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /home/pjw1210/anaconda3/envs/tfasr075cpu/lib/python3.8/site-packages/tensorflow_text/python/metrics/_text_similarity_metric_ops.so: undefined symbol: _ZNK10tensorflow8OpKernel11TraceStringERKNS_15OpKernelContextEb

So I install tensorflow-text 2.3.0 instead of tensorflow-text 2.4.3

And then It works finally!

I want to remain this issue 'open' Becuase tensorflow 2.4, tensorflow_io 0.17.0, tensorflow-text 2.4.3 are installed automatically at pip install . now How think about it?

nglehuy commented 3 years ago

@JeongWoonPark I'll add the dependencies to readme, the pip will install the latest versions but the requirements still accept tf2.3.2 We can close this issue.