export to tflite using python37 venv

[wasertech]

This PR makes available a new venv using python3.7 to export your checkpoints to the TFLite format.

Instead of using the image' system python (3.8), use python 3.7 from venv at /tflite-venv/bin/python.

root@65576a3b3080:/code# /tflite-venv/bin/python -m coqui_stt_training.export --alphabet_config_path /mnt/models/alphabet.txt --scorer_path /mnt/lm/kenlm.scorer --feature_cache /mnt/sources/feature_cache --n_hidden 2048 --beam_width 500 --lm_alpha 1.6383103526118539 --lm_beta 0.1291025653672666 --load_evaluate best --checkpoint_dir /mnt/checkpoints/ --export_dir /mnt/models/ --export_tflite true --export_author_id CommonVoice-FR-Team --export_model_version 0.9 --export_contact_info https://discourse.mozilla.org/c/voice/fr --export_license MIT-0 --export_language fr-FR --export_min_stt_version 1.0.0 --export_max_stt_version 1.4.0 --export_model_name cv-fr-tflite
I Exporting the model...
I Loading best validating checkpoint from /mnt/checkpoints/best_dev-0
I Loading variable from checkpoint: cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/bias
I Loading variable from checkpoint: cudnn_lstm/rnn/multi_rnn_cell/cell_0/cudnn_compatible_lstm_cell/kernel
I Loading variable from checkpoint: layer_1/bias
I Loading variable from checkpoint: layer_1/weights
I Loading variable from checkpoint: layer_2/bias
I Loading variable from checkpoint: layer_2/weights
I Loading variable from checkpoint: layer_3/bias
I Loading variable from checkpoint: layer_3/weights
I Loading variable from checkpoint: layer_5/bias
I Loading variable from checkpoint: layer_5/weights
I Loading variable from checkpoint: layer_6/bias
I Loading variable from checkpoint: layer_6/weights
I Models exported at /mnt/models/
I Model metadata file saved to /mnt/models/CommonVoice-FR-Team_cv-fr-tflite_0.9.md. Before submitting the exported model for publishing make sure all information in the metadata file is correct, and complete the URL fields.

See my logs here. See #2220 for context.

[reuben]

Good idea, thanks for the PR! We should add some instructions to the error message for this case: https://github.com/coqui-ai/STT/blob/178b4f146fdd260fbfb0cbb109343bbd6acdbc55/training/coqui_stt_training/export.py#L131-L141

[wasertech]

May 22 saw changes to protobuff introduced to python wihich means we need to regenerate our protos. Workaround are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

See the error message:

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tflite-venv/lib/python3.7/site-packages/coqui_stt_training/export.py", line 13, in <module>
    import tensorflow as tf
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow/__init__.py", line 102, in <module>
    from tensorflow_core import *
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/__init__.py", line 28, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "<frozen importlib._bootstrap>", line 1019, in _handle_fromlist
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow/__init__.py", line 50, in __getattr__
    module = self._load()
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow/__init__.py", line 44, in _load
    module = _importlib.import_module(self.__name__)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/python/__init__.py", line 52, in <module>
    from tensorflow.core.framework.graph_pb2 import *
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/core/framework/graph_pb2.py", line 16, in <module>
    from tensorflow.core.framework import node_def_pb2 as tensorflow_dot_core_dot_framework_dot_node__def__pb2
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/core/framework/node_def_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/core/framework/tensor_shape_pb2.py", line 42, in <module>
    serialized_options=None, file=DESCRIPTOR),
  File "/tflite-venv/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

[reuben]

It's still missing some linting errors, do you not have the pre-commit hook installed? Instructions here: https://github.com/coqui-ai/STT/blob/main/CONTRIBUTING.rst#code-linting

[reuben]

Ah, nevermind, looks like those are from the other PR, which means the answer to my question is yes :P

[wasertech]

Ah, nevermind, looks like those are from the other PR, which means the answer to my question is yes :P

I'll make sure to hook it up!

EDIT: I've installed it but I won't retroactively lint the code, is there something I can do?

[alpoktem]

Hi, I'm stuck in something similar and will appreciate a lot some support. My model trained well and performed tests but didn't export anything to the export_dir. The official docker image didn't come with the tflite-venv as @wasertech suggests to use so I created myself following this script inside the docker:

apt update
apt install software-properties-common
add-apt-repository ppa:deadsnakes/ppa
apt install python3.7
apt install python3.7-venv
cd /
python3.7 -m venv tflite-venv
/tflite-venv/bin/pip3.7 install coqui_stt_training 
/tflite-venv/bin/pip3.7 install protobuf==3.20.1
/tflite-venv/bin/pip3.7 install torch
export PATH=$PATH:/tflite-venv/bin
/tflite-venv/bin/python3.7 -m coqui_stt_training.export --checkpoint_dir /STT/stt-data/hau/checkpoints/hau-from-en-v0.1 --export_dir /STT/stt-data/hau/exported-models/hau-from-en-v0.1

Although, I'm still getting this:

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tflite-venv/lib/python3.7/site-packages/coqui_stt_training/export.py", line 317, in <module>
    main()
  File "/tflite-venv/lib/python3.7/site-packages/coqui_stt_training/export.py", line 305, in main
    export()
  File "/tflite-venv/lib/python3.7/site-packages/coqui_stt_training/export.py", line 136, in export
    tflite_model = converter.convert()
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/lite/python/lite.py", line 983, in convert
    **converter_kwargs)
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/lite/python/convert.py", line 449, in toco_convert_impl
    enable_mlir_converter=enable_mlir_converter)
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/lite/python/convert.py", line 200, in toco_convert_protos
    raise ConverterError("See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: See console for info.
/bin/bash: toco_from_protos: command not found

I checked and toco_from_protos can be reached from the docker's bash as I added /tflite-venv/bin to the PATH.

For your information, this is how I pulled the docker image in the first place:

docker run  -it --entrypoint /bin/bash --name stt-train --gpus all --mount type=bind,source="$(pwd)"/stt-data,target=/STT/stt-data  ghcr.io/coqui-ai/stt-train

[alpoktem]

Hi, I'm stuck in something similar and will appreciate a lot some support. My model trained well and performed tests but didn't export anything to the export_dir. The official docker image didn't come with the tflite-venv as @wasertech suggests to use so I created myself following this script inside the docker:

apt update
apt install software-properties-common
add-apt-repository ppa:deadsnakes/ppa
apt install python3.7
apt install python3.7-venv
cd /
python3.7 -m venv tflite-venv
/tflite-venv/bin/pip3.7 install coqui_stt_training 
/tflite-venv/bin/pip3.7 install protobuf==3.20.1
/tflite-venv/bin/pip3.7 install torch
export PATH=$PATH:/tflite-venv/bin
/tflite-venv/bin/python3.7 -m coqui_stt_training.export --checkpoint_dir /STT/stt-data/hau/checkpoints/hau-from-en-v0.1 --export_dir /STT/stt-data/hau/exported-models/hau-from-en-v0.1

Although, I'm still getting this:

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tflite-venv/lib/python3.7/site-packages/coqui_stt_training/export.py", line 317, in <module>
    main()
  File "/tflite-venv/lib/python3.7/site-packages/coqui_stt_training/export.py", line 305, in main
    export()
  File "/tflite-venv/lib/python3.7/site-packages/coqui_stt_training/export.py", line 136, in export
    tflite_model = converter.convert()
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/lite/python/lite.py", line 983, in convert
    **converter_kwargs)
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/lite/python/convert.py", line 449, in toco_convert_impl
    enable_mlir_converter=enable_mlir_converter)
  File "/tflite-venv/lib/python3.7/site-packages/tensorflow_core/lite/python/convert.py", line 200, in toco_convert_protos
    raise ConverterError("See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: See console for info.
/bin/bash: toco_from_protos: command not found

I checked and toco_from_protos can be reached from the docker's bash as I added /tflite-venv/bin to the PATH.

For your information, this is how I pulled the docker image in the first place:

docker run  -it --entrypoint /bin/bash --name stt-train --gpus all --mount type=bind,source="$(pwd)"/stt-data,target=/STT/stt-data  ghcr.io/coqui-ai/stt-train

resolved for now by adding this line to /tflite-venv/lib/python3.7/site-packages/tensorflow_core/lite/python/convert.py

_toco_from_proto_bin = "/tflite-venv/bin/toco_from_protos"

[wasertech]

@alpoktem the virtual environment was officially added to the image with version 1.4.0 so your trick isn’t necessary anymore. Just pull the latest stable image.

Also it’s bad practice to comment about your issues under someone else’s closed PR. Either open a new discussion or a ticket next time. We would have seen your message and would have tell you to pull the image from main to get the venv for tflite export.

[alpoktem]

Hi, Hola,

Estic de vacances fins al 12 de Gener. Si necessites alguna cosa urgent, si us plau, escriu a @.***

I am taking time off until 12th of January. If you need anything urgent please write to @.***

Estoy de vacaciones hasta el 13 de Enero. Si necesitas algo urgente, por favor, escribe a @.***

-- 

Alp Öktem Col·lectivaT, SCCL web:https://collectivat.cat/  @.*** @collectivat