Facing problem with installation on python3i

[vhiwase]

I am facing problem in installation of DeepSpeech2 model in python3.5.2. Speciallfically problem comes from DeepSpeech-develop/decoders/swig/swig_decoders.py which could not find _swig_decoders.py due to scorer.h:9:33: fatal error: lm/enumerate_vocab.hh: No such file or directory

Error messages as follows while doing setup from /DeepSpeech-develop/decoders/swig/setup.py: DeepSpeech-develop/decoders/swig$ sudo python3 setup.py install scorer.h:22: Warning 401: Nothing known about base class 'lm::EnumerateVocab'. Ignored. running install running bdist_egg running egg_info creating swig_decoders.egg-info writing top-level names to swig_decoders.egg-info/top_level.txt writing dependency_links to swig_decoders.egg-info/dependency_links.txt writing swig_decoders.egg-info/PKG-INFO writing manifest file 'swig_decoders.egg-info/SOURCES.txt' reading manifest file 'swig_decoders.egg-info/SOURCES.txt' writing manifest file 'swig_decoders.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-3.5 copying swig_decoders.py -> build/lib.linux-x86_64-3.5 running build_ext building '_swig_decoders' extension creating build/temp.linux-x86_64-3.5 x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I. -Ikenlm -Iopenfst-1.6.3/src/include -IThreadPool -I/usr/include/python3.5m -c decoders_wrap.cxx -o build/temp.linux-x86_64-3.5/decoders_wrap.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_ZLIB -DHAVE_BZLIB -DHAVE_XZLIB cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ In file included from decoders_wrap.cxx:2806:0: scorer.h:9:33: fatal error: lm/enumerate_vocab.hh: No such file or directory compilation terminated. x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I. -Ikenlm -Iopenfst-1.6.3/src/include -IThreadPool -I/usr/include/python3.5m -c decoder_utils.cpp -o build/temp.linux-x86_64-3.5/decoder_utils.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_ZLIB -DHAVE_BZLIB -DHAVE_XZLIB error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++

When I am trying to run infer.py form python3 it is giving me following error: python3 infer.py Traceback (most recent call last): File "infer.py", line 14, in from model_utils.model import DeepSpeech2Model File "/DeepSpeech-develop/model_utils/model.py", line 20, in from decoders.swig_wrapper import Scorer File "/DeepSpeech-develop/decoders/swig_wrapper.py", line 6, in from .swig import swig_decoders File "/DeepSpeech-develop/decoders/swig/swig_decoders.py", line 13, in from . import _swig_decoders

[vhiwase]

Found Solution:

change setup.sh as follows:

!/usr/bin/env bash

if [ ! -d kenlm ]; then git clone https://github.com/luotao1/kenlm.git echo -e "\n" fi

if [ ! -d openfst-1.6.3 ]; then echo "Download and extract openfst ..." wget http://www.openfst.org/twiki/pub/FST/FstDownload/openfst-1.6.3.tar.gz tar -xzvf openfst-1.6.3.tar.gz echo -e "\n" fi

if [ ! -d ThreadPool ]; then git clone https://github.com/progschj/ThreadPool.git echo -e "\n" fi

echo "Install decoders ..." sudo python3 setup.py install --num_processes 4

---

Then run command sh install.sh copy /DeepSpeech-develop/decoders/swig/build/lib.linux-x86_64-3.5/_swig_decoders.cpython-35m-x86_64-linux-gnu.so to /DeepSpeech-develop/decoders/swig/_swig_decoders.cpython-35m-x86_64-linux-gnu.so

[inishchith]

@VaibhavHiwase Were you able to get this working on python3.5?

this is what I get when trying to execute examples/deploy_demo/run_english_demo_server.sh:

2020-02-03 20:16:35,089-INFO: begin to initialize the external scorer for decoding
Traceback (most recent call last):
  File "deploy/demo_server.py", line 169, in <module>
    main()
  File "deploy/demo_server.py", line 165, in main
    start_server()
  File "deploy/demo_server.py", line 116, in start_server
    vocab_list)
  File "/home/Nishchith/test_server/DeepSpeech/deploy/../model_utils/model.py", line 478, in init_ext_scorer
    language_model_path, vocab_list)
  File "/home/Nishchith/test_server/DeepSpeech/deploy/../decoders/swig_wrapper.py", line 23, in __init__
    swig_decoders.Scorer.__init__(self, alpha, beta, model_path, vocabulary)
  File "/usr/local/lib/python3.6/dist-packages/swig_decoders-1.1-py3.6-linux-x86_64.egg/swig_decoders.py", line 1140, in __init__
    _swig_decoders.Scorer_swiginit(self, _swig_decoders.new_Scorer(alpha, beta, lm_path, vocabulary))
TypeError: in method 'new_Scorer', argument 4 of type 'std::vector< std::string,std::allocator< std::string > > const &'
Failed in starting demo server!

Please let me know in case you too encountered this or else if I'm missing something out.

Thanks!

[sungyihsun]

@inishchith I have the same problem. Did you solve it?

[inishchith]

@sungyihsun Sorry, No. I had to continue with python2

[VaibhavAbhimanyooHiwase]

@sungyihsun ,@inishchith I have created a git repo for you guys to follow the steps that I had gone through when I was working on this model. Hope this will help you.

Git repo link: https://github.com/VaibhavAbhimanyooHiwase/DeepSpeech2

Feel free to contact me at hiwaseva@rknec.edu, hiwase.vaibhav@gmail.com or connect me through LinkedIn by https://www.linkedin.com/in/vaibhav-hiwase

[inishchith]

@VaibhavAbhimanyooHiwase Thanks for sharing this. I'll have a look at it in some time.

Thanks!

[zh794390558]

vocab_list = [chars for chars in data_generator.vocab_list] in line 169.

[detrin]

Found Solution:

change setup.sh as follows:

!/usr/bin/env bash

if [ ! -d kenlm ]; then git clone https://github.com/luotao1/kenlm.git echo -e "\n" fi

if [ ! -d openfst-1.6.3 ]; then echo "Download and extract openfst ..." wget http://www.openfst.org/twiki/pub/FST/FstDownload/openfst-1.6.3.tar.gz tar -xzvf openfst-1.6.3.tar.gz echo -e "\n" fi

if [ ! -d ThreadPool ]; then git clone https://github.com/progschj/ThreadPool.git echo -e "\n" fi

echo "Install decoders ..." sudo python3 setup.py install --num_processes 4

Then run command sh install.sh copy /DeepSpeech-develop/decoders/swig/build/lib.linux-x86_64-3.5/_swig_decoders.cpython-35m-x86_64-linux-gnu.so to /DeepSpeech-develop/decoders/swig/_swig_decoders.cpython-35m-x86_64-linux-gnu.so

I am trying to use https://github.com/dangvansam98/demo_vietasr and this helped me a lot. I have to note that the proper script would be now

#!/usr/bin/env bash

if [ ! -d kenlm ]; then
    git clone https://github.com/kpu/kenlm.git
    echo -e "\n"
fi

if [ ! -d openfst-1.6.3 ]; then
    echo "Download and extract openfst ..."
    wget http://www.openfst.org/twiki/pub/FST/FstDownload/openfst-1.6.3.tar.gz
    tar -xzvf openfst-1.6.3.tar.gz
    echo -e "\n"
fi

if [ ! -d ThreadPool ]; then
    git clone https://github.com/progschj/ThreadPool.git
    echo -e "\n"
fi

echo "Install decoders ..."
sudo python3 setup.py install --num_processes 4

because kpu now owns KenLM.