Closed darlene898 closed 2 years ago
Because the DeepSpeech project is not supporting Python bindings for Python 3.8 and later for 64 bit arm systems (aarch64), e.g. used on the Raspberry Pi 4 running Ubuntu server 20.04, DeepSpeech and DeepSpeech-TFLite must be compiled manually.
You have to install Bazel 3.1.0 first:
wget https://github.com/bazelbuild/bazel/releases/download/3.7.2/bazel-3.7.2-linux-arm64
chmod 755 bazel-3.7.2-linux-arm64
mv bazel-3.7.2-linux-arm64 /usr/local/bin/bazel
cd "/usr/local/lib/bazel/bin" && curl -fLO https://releases.bazel.build/3.1.0/release/bazel-3.1.0-linux-x86_64 && chmod +x bazel-3.1.0-linux-x86_64
Install Python 3.8 locally and set it as default (this should be default on Ubuntu 20.04 and can be skipped):
git clone https://github.com/pyenv/pyenv.git ~/.pyenv
# Follow instructions on https://github.com/pyenv/pyenv
sudo apt-get install libbz2-dev libssl-dev libreadline-dev libsqlite3-dev
pyenv install 3.8.10
pyenv local 3.8.10
git clone https://github.com/mozilla/DeepSpeech.git
git checkout v0.9.3
git submodule sync tensorflow/
git submodule update --init tensorflow/
cd tensorflow
Configure:
When configuring TensorFlow use "-march=armv8-a+crc -Wno-sign-compare" when you are asked:
./configure
Compile:
NOTE: This is targeting TFLite, but in the comments also the compilation for standard DeepSpeech is given.
# Use tmux to keep the process running, when logged in over ssh, if doing this locally, you can skip it
tmux
bazel clean
# For non TFLite version:
#bazel --host_jvm_args=-Xmx6000m build --workspace_status_command="bash native_client/bazel_workspace_status_cmd.sh" --local_cpu_resources=1 -c opt --copt=-O3 --copt="-D_GLIBCXX_USE_CXX11_ABI=0" --config=monolithic --config=nogcp --config=nohdfs --config=nonccl --copt=-fvisibility=hidden --config=noaws --copt=-ftree-vectorize --copt=-funsafe-math-optimizations --copt=-ftree-loop-vectorize --copt=-fomit-frame-pointer //native_client:libdeepspeech.so
# For TFLite version
bazel --host_jvm_args=-Xmx6000m build --workspace_status_command="bash native_client/bazel_workspace_status_cmd.sh" --local_cpu_resources=1 -c opt --copt=-O3 --copt="-D_GLIBCXX_USE_CXX11_ABI=0" --define=runtime=tflite --config=monolithic --config=nogcp --config=nohdfs --config=nonccl --copt=-fvisibility=hidden --config=noaws --copt=-ftree-vectorize --copt=-funsafe-math-optimizations --copt=-ftree-loop-vectorize --copt=-fomit-frame-pointer //native_client:libdeepspeech.so //native_client:tflite
cd ../native_client
sudo apt-get install -y libsox-dev libpng-dev libgsm1-dev libmagic-dev libltdl-dev liblzma-dev libbz2-dev swig
make deepspeech
# Do not execute `PREFIX=/usr/local sudo make install` like instructed in the manual, otherwise the libdeepspeech.so will not be included in the Python wheel
Apply this patch to use the correct naming for the Python wheel:
diff --cc native_client/definitions.mk
index a3af0970,72c12e3a..00000000
--- a/native_client/definitions.mk
+++ b/native_client/definitions.mk
@@@ -51,7 -52,11 +52,11 @@@ SOX_LDFLAGS := `pkg-config --libs s
endif # OS others
PYTHON_PACKAGES := numpy${NUMPY_BUILD_VERSION}
ifeq ($(OS),Linux)
+ ifeq ($(PROCESSOR),x86_64)
-PYTHON_PLATFORM_NAME := --plat-name manylinux1_x86_64
+PYTHON_PLATFORM_NAME ?= --plat-name manylinux1_x86_64
+ else
-PYTHON_PLATFORM_NAME := --plat-name linux_${PROCESSOR}
++PYTHON_PLATFORM_NAME ?= --plat-name linux_${PROCESSOR}
+ endif
endif
endif
NOTE: This is targeting TFLite, but in the comments also the compilation for standard DeepSpeech is given.
cd python
# For non TFLite version
# make bindings
# For TFLite version
make SETUP_FLAGS="--project_name deepspeech_tflite" bindings
pip install dist/deepspeech*.whl
Due to the change in dependencies, especially SWIG being shifted from https://community-tc.services.mozilla.com/ to ds-swig pre-built package from Mozilla-DS, a new patch is documented below (a major chunk of the fix is the same as above):
You first have to install Bazel 3.1.0. do not install the latest version of Bazel (5.1 as of time of this writing) it is not compatible with tensorflow build.
$ sudo wget https://github.com/bazelbuild/bazel/releases/download/3.7.2/bazel-3.7.2-linux-arm64
$ chmod 755 bazel-3.7.2-linux-arm64
$ mv bazel-3.7.2-linux-arm64 /usr/local/bin/bazel
$ mkdir /usr/local/lib/bazel && mkdir /usr/local/lib/bazel/bin
$ cd "/usr/local/lib/bazel/bin"
$ curl -fLO https://releases.bazel.build/3.1.0/release/bazel-3.1.0-linux-x86_64 && chmod +x bazel-3.1.0-linux-x86_64
Upgrade python to python3.8 or 3.9. Set the path variables.
$ sudo apt-get install python3.9
$ export PATH=$PATH:/usr/local/bin
$ git clone https://github.com/mozilla/DeepSpeech.git
$ git checkout v0.9.3
$ git submodule sync tensorflow/
$ git submodule update --init tensorflow/
$ cd tensorflow
When configuring TensorFlow use "-march=armv8-a+crc -Wno-sign-compare" when you are asked:
$ ./configure
$ sudo apt update
$ sudo apt install python-is-python3
# Use tmux to keep the process running, when logged in over ssh, if doing # this locally, you can skip it
$ tmux
$ bazel clean
# For non TFLite version:
# bazel --host_jvm_args=-Xmx6000m build --workspace_status_command="bash native_client/bazel_workspace_status_cmd.sh" --local_cpu_resources=1 -c opt --copt=-O3 --copt="-D_GLIBCXX_USE_CXX11_ABI=0" --config=monolithic --config=nogcp --config=nohdfs --config=nonccl --copt=-fvisibility=hidden --config=noaws --copt=-ftree-vectorize --copt=-funsafe-math-optimizations --copt=-ftree-loop-vectorize --copt=-fomit-frame-pointer //native_client:libdeepspeech.so
# For TFLite version
$ bazel --host_jvm_args=-Xmx6000m build --workspace_status_command="bash native_client/bazel_workspace_status_cmd.sh" --local_cpu_resources=1 -c opt --copt=-O3 --copt="-D_GLIBCXX_USE_CXX11_ABI=0" --define=runtime=tflite --config=monolithic --config=nogcp --config=nohdfs --config=nonccl --copt=-fvisibility=hidden --config=noaws --copt=-ftree-vectorize --copt=-funsafe-math-optimizations --copt=-ftree-loop-vectorize --copt=-fomit-frame-pointer //native_client:libdeepspeech.so //native_client:tflite
$ cd ../native_client
$ sudo apt-get install -y libsox-dev libpng-dev libgsm1-dev libmagic-dev libltdl-dev liblzma-dev libbz2-dev
$ sudo make deepspeech
# Do not install swig as given in GitHub Documentation.
# Do not execute `PREFIX=/usr/local sudo make install` like instructed in the manual, otherwise the libdeepspeech.so will not be included in the Python wheel
$ sudo apt update
$ sudo apt install python3-testresources
$ sudo pip3 install numpy –upgrade
$ sudo apt-get install byacc
$ sudo apt-get remove swig
$ sudo git clone https://github.com/swig/swig
$ cd swig/
$ sudo sh autogen.sh
$ sudo ./configure --prefix=<path/to/deepspeech>/native_client/ds-swig/ --program-prefix=ds-
$ sudo make -j
$ sudo make -j install
$ cd ../ds-swig/bin && ln -s ds-swig swig
$ cd /usr/bin
$ sudo ln -s <path/to/deepspeech>/native_client/ds-swig/bin/ds-swig swig
$ swig --version
# Check SWIG install. Should return: SWIG Version 4.1.0
Apply the following patches:
$ cd python
# For non TFLite version
# make bindings
# For TFLite version
$ make SETUP_FLAGS="--project_name deepspeech_tflite" bindings
$ pip install dist/deepspeech*.whl
There was already documentation covering that. DeepSpeech is not maintained anymore, there is no point in supporting newer versions of Python. Please look at Coqui https://github.com/coqui-ai/STT/
@RaunakSRajpal will this update (from Aug 23, 2022) work on aarch64 (Ubuntu 20.04) ?
Well, then I will document the patch at least here or somewhere else if embedded arm devices on recent Python versions are not in the current scope of the Deepspeech project so that other programmers can still use it. If it will not be merged, this can be closed.
Originally posted by @kaoh in https://github.com/mozilla/DeepSpeech/issues/3663#issuecomment-868854706