search

NVIDIA / OpenSeq2Seq

Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP
https://nvidia.github.io/OpenSeq2Seq
Apache License 2.0
1.54k stars 369 forks source link

Language Model Build Error for Speech Recognition #351

Closed SibtainRazaJamali closed 5 years ago

SibtainRazaJamali commented 5 years ago

I am getting error while building ctc_decoder_with_lm language model.Please specify which versions of tensorflow ,bazel is required to build language model. There is some error in generate_trie.cpp file as well. Kindly check it I have used tensorflow 1.10,1.11,1.12 and bazel version from 15 to 20 (all tried one by one). Previously there was a version mismatch of bazel and tensorflow but now there is error in generate_trie.cpp file. image

borisgin commented 5 years ago

1)Please clean .cache/bazel in the home directory 2)if you building Tensorflow with NCCL, then check the location of libnccl.so ( e.g. $ locate libnccl.so, and make sure that this location is in $LD_LIB_PATH) 3) rerun .configure and set location of libnccl.so

SibtainRazaJamali commented 5 years ago

Sorry it is still not clear to me.I have followed each step in your documentation. But i am still getting errors while building LM. Kindly show me an easy way to build language model.

borisgin commented 5 years ago

Can you attach the complete log please?

SibtainRazaJamali commented 5 years ago

image

borisgin commented 5 years ago

Can you attach the following information please: $ lsb_release -a $ echo $LD_LIBRARY_PATH $ locate libcuda.so $ locate libcudnn.so $ locate libnccl.so $ bazel version $ cd tensorflow; git status Please also attach the file called .tf_configure.bazelrc. It should be in tensorflow directory

SibtainRazaJamali commented 5 years ago

There is no file of this name .tf_configure.bazelrc. image image image

borisgin commented 5 years ago

I would recommend first to upgrade to CUDA10, cudnn, and install cudnn. For this remove previous installation $ sudo apt-get purge cuda $ sudo apt-get purge cuda-toolkit-* $ sudo apt-get purge libcudnn7 $ sudo apt-get purge libcudnn7-dev and then

  1. install cuda10 from https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1804&target_type=debnetwork $ sudo apt-get install cuda
  2. update LD_LIBRARY_PATH and PATH in .bashrc
  3. install cudnn 7 for cuda10 https://developer.nvidia.com/cudnn $sudo apt-get install libcudnn7 $sudo apt-get install libcudnn7-dev
  4. install nccl2 https://developer.nvidia.com/nccl $ apt-get install libnccl2 $ apt-get install libnccl-dev check where libnccl.so is located and make sure that LD_LIBRARY_PATH points to this location

Next you need python3.5 or 3.6 from Anaconda: wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh and make sure that path to is added to .bashrc

remove ~.cache/bazel and do $ bazel clean