Java Build Error - OpenBLAS libblas.a

[GABowers]

I'm running Ubuntu in VirtualBox. I cloned and compiled kaldi. I've run a couple examples and it works.

I tried cloning vosk and following the instructions at (https://github.com/alphacep/vosk-api/tree/master/java), but I get a bunch of "file not found" errors. All but the last one ARE at their paths, but for the last one(OpenBLAS/install/lib/libblas.a), the file is not in my OpenBLAS folder from kaldi, even with a successful kaldi build.

I also tried cloning OpenBLAS itself (from https://github.com/xianyi/OpenBLAS) and compiling it, but the file in question wasn't created there either.

[nshmyrev]

You didn't compile kaldi with openblas-clapack. You can find detailed instructions here:

https://github.com/alphacep/vosk-api/blob/67de30908bd0b3222453f9cce38870d1beedc856/travis/Dockerfile.manylinux#L25

[GABowers]

Alright, I made a couple modifications to the last bit to fit my file structure. I have to delete the original OpenBLAS folder and it gets to near the end. It then says I need to specify the clapack directory, but after I set it, it seems to be looking somewhere else entirely.

[nshmyrev]

Mathlib is OPENBLAS_CLAPACK, not CLAPACK.

[GABowers]

Unfortunately it didn't want to recognize that when I was building from kaldi's repo directly: So I created the /opt folder in the vosk-api folder and used the entire third bit from your script (with your fork of kaldi). Had to modify the paths, and change the exec to delete, but now it's giving a different error - "recipe for target 'hmm' failed" What I'm seeing online (https://github.com/kaldi-asr/kaldi/issues/2697) implies it's due to different compilers when compiling /tools and /src, but I don't see anything specifying compiler at all.

Just to be clear, I'm:

1. cloning the vosk-api repo
2. Creating the opt folder
3. Running the commands on the right in the second image

[nshmyrev]

What is your OS and GCC version? It is better to provide build log file as a text, it is not a great idea to paste screenshots on technical issues.

[GABowers]

Sorry, I was using screenshots because I'm working with a VM. Didn't occur to me until now to just do this from within it. The OS is Ubuntu 18.04.5 LTS, and GCC version is 7.5.0.

The command used within the vosk-api/opt folder was:

git clone -b lookahead-1.8.0 --single-branch https://github.com/alphacep/kaldi \ && cd /home/gab/Documents/vosk-api/opt/kaldi/tools \ && git clone -b v0.3.13 --single-branch https://github.com/xianyi/OpenBLAS \ && git clone -b v3.2.1 --single-branch https://github.com/alphacep/clapack \ && make -C OpenBLAS ONLY_CBLAS=1 DYNAMIC_ARCH=1 USE_LOCKING=1 USE_THREAD=0 all >&log0 \ && make -C OpenBLAS PREFIX=$(pwd)/OpenBLAS/install install >&log1 \ && mkdir -p clapack/BUILD && cd clapack/BUILD && cmake .. && make -j 10 && find . -name ".a" | xargs cp -t ../../OpenBLAS/install/lib \ && cd /home/gab/Documents/vosk-api/opt/kaldi/tools \ && git clone --single-branch https://github.com/alphacep/openfst openfst \ && cd openfst \ && autoreconf -i \ && ./configure --prefix=/home/gab/Documents/vosk-api/opt/kaldi/tools/openfst --enable-static --enable-shared --enable-far --enable-ngram-fsts --enable-lookahead-fsts --with-pic --disable-bin \ && make -j 10 && make install >&log2 \ && cd /home/gab/Documents/vosk-api/opt/kaldi/src \ && ./configure --mathlib=OPENBLAS_CLAPACK --shared --use-cuda=no \ && make -j 10 online2 lm rnnlm >&log3 \ && find /home/gab/Documents/vosk-api/opt/kaldi -name ".o" -delete

The log files are below.

log0.txt log1.txt log2.txt log3.txt

[nshmyrev]

From the log it goes out of memory on your VM. Try to replace -j 10 with -j 2 in the last make

[GABowers]

Alright. Here's the current command:

git clone -b lookahead-1.8.0 --single-branch https://github.com/alphacep/kaldi \ && cd /home/gab/Documents/vosk-api/opt/kaldi/tools \ && git clone -b v0.3.13 --single-branch https://github.com/xianyi/OpenBLAS \ && git clone -b v3.2.1 --single-branch https://github.com/alphacep/clapack \ && make -C OpenBLAS ONLY_CBLAS=1 DYNAMIC_ARCH=1 USE_LOCKING=1 USE_THREAD=0 all >&log0 \ && make -C OpenBLAS PREFIX=$(pwd)/OpenBLAS/install install >&log1 \ && mkdir -p clapack/BUILD && cd clapack/BUILD && cmake .. && make -j 10 && find . -name ".a" | xargs cp -t ../../OpenBLAS/install/lib \ && cd /home/gab/Documents/vosk-api/opt/kaldi/tools \ && git clone --single-branch https://github.com/alphacep/openfst openfst \ && cd openfst \ && autoreconf -i \ && ./configure --prefix=/home/gab/Documents/vosk-api/opt/kaldi/tools/openfst --enable-static --enable-shared --enable-far --enable-ngram-fsts --enable-lookahead-fsts --with-pic --disable-bin \ && make -j 2 && make install >&log2 \ && cd /home/gab/Documents/vosk-api/opt/kaldi/src \ && ./configure --mathlib=OPENBLAS_CLAPACK --shared --use-cuda=no \ && make -j 2 online2 lm rnnlm >&log3 \ && find /home/gab/Documents/vosk-api/opt/kaldi -name ".o" -delete

And the relevant log file: log3.txt

[GABowers]

Okay, I reduced it further to just one core, and increased my VM memory to 8 GB and it worked.

git clone -b lookahead-1.8.0 --single-branch https://github.com/alphacep/kaldi \ && cd /home/gab/Documents/vosk-api/opt/kaldi/tools \ && git clone -b v0.3.13 --single-branch https://github.com/xianyi/OpenBLAS \ && git clone -b v3.2.1 --single-branch https://github.com/alphacep/clapack \ && make -C OpenBLAS ONLY_CBLAS=1 DYNAMIC_ARCH=1 USE_LOCKING=1 USE_THREAD=0 all \ && make -C OpenBLAS PREFIX=$(pwd)/OpenBLAS/install install \ && mkdir -p clapack/BUILD && cd clapack/BUILD && cmake .. && make -j 1 && find . -name "*.a" | xargs cp -t ../../OpenBLAS/install/lib \ && cd /home/gab/Documents/vosk-api/opt/kaldi/tools \ && git clone --single-branch https://github.com/alphacep/openfst openfst \ && cd openfst \ && autoreconf -i \ && ./configure --prefix=/home/gab/Documents/vosk-api/opt/kaldi/tools/openfst --enable-static --enable-shared --enable-far --enable-ngram-fsts --enable-lookahead-fsts --with-pic --disable-bin \ && make -j 1 && make install >&log2 \ && cd /home/gab/Documents/vosk-api/opt/kaldi/src \ && ./configure --mathlib=OPENBLAS_CLAPACK --shared --use-cuda=no \ && make -j 1 online2 lm rnnlm >&log3 \ && find /home/gab/Documents/vosk-api/opt/kaldi -name "*.o" -name ".o" -delete

However, resuming the steps for java still shows the same errors. I set JAVA_HOME and KALDI_ROOT in the /etc/environments file:

JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64" KALDI_ROOT=/home/gab/Documents/vosk-api/opt/kaldi"

Output is:

gab@gab-VirtualBox:[removed tilde]/Documents/vosk-api/java$ make g++ -shared -o libvosk_jni.so -I/usr/lib/jvm/java-8-openjdk-amd64/include -I/usr/lib/jvm/java-8-openjdk-amd64/include/linux -I/home/gab/Documents/vosk-api/opt/kaldi"/src -I/home/gab/Documents/vosk-api/opt/kaldi"/tools/openfst/include -I../src -g -O2 -DPIC -fPIC -Wno-unused-function -std=c++17 vosk_wrap.cc ../src/kaldi_recognizer.cc ../src/kaldi_recognizer.h ../src/language_model.cc ../src/language_model.h ../src/model.cc ../src/model.h ../src/spk_model.cc ../src/spk_model.h ../src/vosk_api.cc ../src/vosk_api.h /home/gab/Documents/vosk-api/opt/kaldi"/src/online2/kaldi-online2.a /home/gab/Documents/vosk-api/opt/kaldi"/src/decoder/kaldi-decoder.a /home/gab/Documents/vosk-api/opt/kaldi"/src/ivector/kaldi-ivector.a /home/gab/Documents/vosk-api/opt/kaldi"/src/gmm/kaldi-gmm.a /home/gab/Documents/vosk-api/opt/kaldi"/src/nnet3/kaldi-nnet3.a /home/gab/Documents/vosk-api/opt/kaldi"/src/tree/kaldi-tree.a /home/gab/Documents/vosk-api/opt/kaldi"/src/feat/kaldi-feat.a /home/gab/Documents/vosk-api/opt/kaldi"/src/lat/kaldi-lat.a /home/gab/Documents/vosk-api/opt/kaldi"/src/lm/kaldi-lm.a /home/gab/Documents/vosk-api/opt/kaldi"/src/hmm/kaldi-hmm.a /home/gab/Documents/vosk-api/opt/kaldi"/src/transform/kaldi-transform.a /home/gab/Documents/vosk-api/opt/kaldi"/src/cudamatrix/kaldi-cudamatrix.a /home/gab/Documents/vosk-api/opt/kaldi"/src/matrix/kaldi-matrix.a /home/gab/Documents/vosk-api/opt/kaldi"/src/fstext/kaldi-fstext.a /home/gab/Documents/vosk-api/opt/kaldi"/src/util/kaldi-util.a /home/gab/Documents/vosk-api/opt/kaldi"/src/base/kaldi-base.a /home/gab/Documents/vosk-api/opt/kaldi"/tools/openfst/lib/libfst.a /home/gab/Documents/vosk-api/opt/kaldi"/tools/openfst/lib/libfstngram.a /home/gab/Documents/vosk-api/opt/kaldi"/tools/OpenBLAS/install/lib/libopenblas.a /home/gab/Documents/vosk-api/opt/kaldi"/tools/OpenBLAS/install/lib/liblapack.a /home/gab/Documents/vosk-api/opt/kaldi"/tools/OpenBLAS/install/lib/libblas.a /home/gab/Documents/vosk-api/opt/kaldi"/tools/OpenBLAS/install/lib/libf2c.a g++: error: /home/gab/Documents/vosk-api/opt/kaldi/src/online2/kaldi-online2.a /home/gab/Documents/vosk-api/opt/kaldi/src/decoder/kaldi-decoder.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/src/ivector/kaldi-ivector.a /home/gab/Documents/vosk-api/opt/kaldi/src/gmm/kaldi-gmm.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/src/nnet3/kaldi-nnet3.a /home/gab/Documents/vosk-api/opt/kaldi/src/tree/kaldi-tree.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/src/feat/kaldi-feat.a /home/gab/Documents/vosk-api/opt/kaldi/src/lat/kaldi-lat.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/src/lm/kaldi-lm.a /home/gab/Documents/vosk-api/opt/kaldi/src/hmm/kaldi-hmm.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/src/transform/kaldi-transform.a /home/gab/Documents/vosk-api/opt/kaldi/src/cudamatrix/kaldi-cudamatrix.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/src/matrix/kaldi-matrix.a /home/gab/Documents/vosk-api/opt/kaldi/src/fstext/kaldi-fstext.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/src/util/kaldi-util.a /home/gab/Documents/vosk-api/opt/kaldi/src/base/kaldi-base.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/tools/openfst/lib/libfst.a /home/gab/Documents/vosk-api/opt/kaldi/tools/openfst/lib/libfstngram.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/tools/OpenBLAS/install/lib/libopenblas.a /home/gab/Documents/vosk-api/opt/kaldi/tools/OpenBLAS/install/lib/liblapack.a: No such file or directory g++: error: /home/gab/Documents/vosk-api/opt/kaldi/tools/OpenBLAS/install/lib/libblas.a /home/gab/Documents/vosk-api/opt/kaldi/tools/OpenBLAS/install/lib/libf2c.a: No such file or directory Makefile:48: recipe for target 'libvosk_jni.so' failed make: *** [libvosk_jni.so] Error 1 gab@gab-VirtualBox:[removed tilde]/Documents/vosk-api/java$

[nshmyrev]

KALDI_ROOT=/home/gab/Documents/vosk-api/opt/kaldi"

You have unmatched quote here, not a good thing

[GABowers]

Okay...I actually just went with Sphinx due to time constraints. This can be closed.

[nshmyrev]

Ok, hopefully once https://github.com/alphacep/vosk-api/issues/358 will be closed java will be easier to use.

[DmitryKey]

Hi I tried to follow these instructions: https://github.com/alphacep/vosk-api/blob/67de30908bd0b3222453f9cce38870d1beedc856/travis/Dockerfile.manylinux#L25

All steps work fine, except this one:

./configure --mathlib=OPENBLAS_CLAPACK --shared --use-cuda=no

which prints:

***configure failed: Unknown --mathlib='OPENBLAS_CLAPACK'. Supported libs: ATLAS CLAPACK MKL OPENBLAS ***

Did anybody face something similar?

[nshmyrev]

Did anybody face something similar?

You need to clone kaldi from a vosk branch of our repo (as done in the docker above).

[DmitryKey]

thanks @nshmyrev I'm trying to do that as in the above instruction file, but getting:

git clone -b lookahead-1.8.0 --single-branch https://github.com/alphacep/kaldi
Cloning into 'kaldi'...
warning: Could not find remote branch lookahead-1.8.0 to clone.
fatal: Remote branch lookahead-1.8.0 not found in upstream origin

[nshmyrev]

the branch is now called vosk, not lookahead-1.8.0

[DmitryKey]

thanks @nshmyrev I will try it out.