search

ccoreilly / vosk-browser

A speech recognition library running in the browser thanks to a WebAssembly build of Vosk
Apache License 2.0
361 stars 61 forks source link

Can't build #64

Open olivierlaporte opened 1 year ago

olivierlaporte commented 1 year ago

Hello I try to reproduce the work produced by arbdevml for issue #49.

I fork the repository. I have docker, make installed.

I reproduce the described steps :

I have the following output : 

#7 [ 4/30] RUN git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579
#7 sha256:d72b762a9137ae3da9126377d52f4ac1e5fb4134afc31851f4a093636254bbdc
#7 0.474 Cloning into '.'...
#7 19.32 fatal: reference is not a tree: 6417ac1dece94783e80dfbac0148604685d27579
#7 ERROR: executor failed running [/bin/sh -c git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579]: exit code: 128
------
 > [ 4/30] RUN git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579:
------
executor failed running [/bin/sh -c git clone -b vosk --single-branch https://github.com/alphacep/kaldi . &&     git checkout 6417ac1dece94783e80dfbac0148604685d27579]: exit code: 128

I think the kaldi project was updated and the git hash does not exists anymore

=> I check the rest of the Dockerfile file and i see that a clone of an inria repository is needed... but this repository seems to be not accessible ?

Someone can help me because i really want to help and have the voice fingerprinting feature with spk model that i have already experiment on python distro ?

simonfn commented 1 year ago

Hello, I got the same issue. I tried to checkout to another commit by replacing "6417ac1dece94783e80dfbac0148604685d27579" to "9af2c5c16389e141f527ebde7ee432a0c1df9fb9", which is the commit done around the same date as the commit from "vosk/". But it fails to apply the "kaldi.patch". 

#10 [ 6/30] RUN git apply -v /tmp/kaldi.patch
#10 1.862 /tmp/kaldi.patch:230: trailing whitespace.
#10 1.862
#10 1.862 /tmp/kaldi.patch:232: trailing whitespace.
#10 1.862
#10 1.862 /tmp/kaldi.patch:269: trailing whitespace.
#10 1.862
#10 1.862 Checking patch src/Makefile...
#10 1.862 error: while searching for:
#10 1.862
#10 1.862 # Reset the default goal, so that the all target will become default
#10 1.862 .DEFAULT_GOAL :=
#10 1.862 all: $(SUBDIRS) matrix/test
#10 1.862       -echo Done
#10 1.862
#10 1.862 mklibdir:
#10 1.862
#10 1.862 error: patch failed: src/Makefile:41
#10 1.862 error: src/Makefile: patch does not apply
#10 1.862 Checking patch src/configure...
#10 1.863 Hunk #1 succeeded at 1026 (offset 4 lines).
#10 1.863 Checking patch src/ivector/ivector-extractor.cc...
#10 1.863 Checking patch src/makefiles/default_rules.mk...
#10 1.863 Checking patch src/makefiles/wasm_clapack.mk...
#10 1.863 Checking patch tools/Makefile...
#10 1.864 error: while searching for:
#10 1.864   # This new OS path is confirmed working on Windows 10 / Cygwin64.
#10 1.864   openfst_add_CXXFLAGS = -g -O2 -Wa,-mbig-obj
#10 1.864 else
#10 1.864   openfst_add_CXXFLAGS = -g -O3 -msse -msse2
#10 1.864 endif
#10 1.864
#10 1.864 openfst-$(OPENFST_VERSION)/Makefile: openfst-$(OPENFST_VERSION)
#10 1.864       cd openfst-$(OPENFST_VERSION)/ && \
#10 1.864         autoreconf -i && \
#10 1.864       ./configure --prefix=`pwd` $(OPENFST_CONFIGURE) CXX="$(CXX)" \
#10 1.864               CXXFLAGS="$(openfst_add_CXXFLAGS) $(CXXFLAGS)" \
#10 1.864
#10 1.864 error: patch failed: tools/Makefile:78
#10 1.864 error: tools/Makefile: patch does not apply
#10 1.864 Checking patch tools/extras/check_dependencies.sh...
#10 1.864 Checking patch tools/openfst.patch...
#10 1.864 /tmp/kaldi.patch:269: new blank line at EOF.
#10 1.864 +
#10 ERROR: executor failed running [/bin/sh -c git apply -v /tmp/kaldi.patch]: exit code: 1
------
 > [ 6/30] RUN git apply -v /tmp/kaldi.patch:
#10 1.864         autoreconf -i && \
#10 1.864       ./configure --prefix=`pwd` $(OPENFST_CONFIGURE) CXX="$(CXX)" \
#10 1.864               CXXFLAGS="$(openfst_add_CXXFLAGS) $(CXXFLAGS)" \
#10 1.864
#10 1.864 error: patch failed: tools/Makefile:78
#10 1.864 error: tools/Makefile: patch does not apply
#10 1.864 Checking patch tools/extras/check_dependencies.sh...
#10 1.864 Checking patch tools/openfst.patch...
#10 1.864 /tmp/kaldi.patch:269: new blank line at EOF.
#10 1.864 +
------
ERROR: failed to solve: executor failed running [/bin/sh -c git apply -v /tmp/kaldi.patch]: exit code: 1
make: *** [Makefile:12: builder] Error 1

Any idea on which commit is the right one ?

olivierlaporte commented 1 year ago

simonfn I have done the same workaround and I have the same error.

I think repositories have changed since last summer and if you have not the builder installed, you cannot release.

ccoreilly commented 1 year ago

I guess I'll have to switch to forks of the repositories which I still have around.

In the meantime you should be able to pull the builder image:

docker pull ghcr.io/ccoreilly/vosk-wasm-builder:0.0.6
simonfn commented 1 year ago

I tried to build the binary from the v.0.0.8 with the docker image from the v.0.0.6, but it fails with :

94 warnings and 1 error generated.
em++: error: '/emsdk/upstream/bin/clang++ -target wasm32-unknown-emscripten -DEMSCRIPTEN -fignore-exceptions -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -D__EMSCRIPTEN_major__=2 -D__EMSCRIPTEN_minor__=0 -D__EMSCRIPTEN_tiny__=29 -D_LIBCPP_ABI_VERSION=2 -Dunix -D__unix -D__unix__ -Werror=implicit-function-declaration -Xclang -iwithsysroot/include/SDL --sysroot=/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -std=c++17 -Wno-unused-function -I/opt/include -I/opt/kaldi/src -I/opt/kaldi/tools/openfst/include -I/io/src -O3 /io/src/../vosk/src/recognizer.cc -c -o /tmp/emscripten_temp_eh0ac_s_/recognizer_2.o' failed (returned 1)
make: *** [Makefile:90: /io/src/../build/release/vosk.bc] Error 1
make: Leaving directory '/io/src'
make: *** [Makefile:19: binary] Error 2

The v.0.0.7 and v.0.0.6 seems to work with the same build environnement with some warnings when compiling .

olivierlaporte commented 1 year ago

I will try on my side and give feedbacks. Thanks a lot @ccoreilly.

ccoreilly commented 1 year ago

@olivierlaporte @simonfn could you test the branch in #66 ?

simonfn commented 1 year ago

@ccoreilly #66 worked for me, although it took quite some times to build the image

olivierlaporte commented 1 year ago

It's also ok on my side. @ccoreilly Thanks you for your quick response and your very quick patch !

And like @simonfn building the image is quite long but it works.

git checkout -b make-builder-reproducible origin/make-builder-reproducible time make builder

Step 31/32 : RUN rm -rf /tmp/libarchive
 ---> Running in 76862ebc8b05
Removing intermediate container 76862ebc8b05
 ---> ebe4d5360042
Step 32/32 : WORKDIR /opt
 ---> Running in 06b31cbed381
Removing intermediate container 06b31cbed381
 ---> f4353dc52c41
Successfully built f4353dc52c41
Successfully tagged ghcr.io/ccoreilly/vosk-wasm-builder:0.0.7
Successfully tagged ghcr.io/ccoreilly/vosk-wasm-builder:latest

time make binary

/emsdk/upstream/emscripten/cache/sysroot/include/c++/v1/__config:979:48: note: expanded from macro '_LIBCPP_DEPRECATED'
#    define _LIBCPP_DEPRECATED __attribute__ ((deprecated))
                                               ^
59 warnings generated.
em++: warning: generating an executable with an object extension (.bc).  If you meant to build an object file please use `-c, `-r`, or `-shared` [-Wemcc]
mkdir -p /io/src/../build/release/
cd /io/src/../build/release/; \
/emsdk/upstream/emscripten/em++ -o vosk.js /io/src/../build/release/vosk.bc --pre-js /io/src/vosk.pre.js --post-js /io/src/vosk.post.js --bind -std=c++17 -s ALLOW_MEMORY_GROWTH=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -s MODULARIZE=1 -s EXPORT_NAME=LoadVosk -s ENVIRONMENT=worker -s EXPORTED_RUNTIME_METHODS=['FS','IDBFS'] -lidbfs.js -O3 -s SINGLE_FILE=1  /opt/kaldi/src/base/kaldi-base.bc /opt/kaldi/src/cudamatrix/kaldi-cudamatrix.bc /opt/kaldi/src/decoder/kaldi-decoder.bc /opt/kaldi/src/feat/kaldi-feat.bc /opt/kaldi/src/fstext/kaldi-fstext.bc /opt/kaldi/src/gmm/kaldi-gmm.bc /opt/kaldi/src/hmm/kaldi-hmm.bc /opt/kaldi/src/ivector/kaldi-ivector.bc /opt/kaldi/src/lat/kaldi-lat.bc /opt/kaldi/src/lm/kaldi-lm.bc /opt/kaldi/src/matrix/kaldi-matrix.bc /opt/kaldi/src/nnet3/kaldi-nnet3.bc /opt/kaldi/src/online2/kaldi-online2.bc /opt/kaldi/src/transform/kaldi-transform.bc /opt/kaldi/src/tree/kaldi-tree.bc /opt/kaldi/src/util/kaldi-util.bc /opt/kaldi/tools/clapack-wasm/CLAPACK-3.2.1/lapack.a /opt/kaldi/tools/clapack-wasm/CLAPACK-3.2.1/libcblaswr.a /opt/kaldi/tools/clapack-wasm/CBLAS/lib/cblas.a /opt/kaldi/tools/clapack-wasm/f2c_BLAS-3.8.0/blas.a /opt/kaldi/tools/openfst/lib/libfst.a /opt/kaldi/tools/openfst/lib/libfstngram.a /opt/lib/libarchive.so
warning: undefined symbol: _ZN5kaldi5rnnlm21RnnlmComputeStateInfoC1ERKNS0_35RnnlmComputeStateComputationOptionsERKNS_5nnet34NnetERKNS_8CuMatrixIfEE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN5kaldi5rnnlm22ReadSparseWordFeaturesERNSt3__213basic_istreamIcNS1_11char_traitsIcEEEEiPNS_12SparseMatrixIfEE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN5kaldi5rnnlm26KaldiRnnlmDeterministicFst5ClearEv (referenced by top-level compiled C/C++ code)
warning: undefined symbol: _ZN5kaldi5rnnlm26KaldiRnnlmDeterministicFstC1EiRKNS0_21RnnlmComputeStateInfoE (referenced by top-level compiled C/C++ code)
warning: undefined symbol: do_fio (referenced by top-level compiled C/C++ code)
warning: undefined symbol: e_wsfe (referenced by top-level compiled C/C++ code)
warning: undefined symbol: i_nint (referenced by top-level compiled C/C++ code)
warning: undefined symbol: popen (referenced by top-level compiled C/C++ code)
warning: undefined symbol: pow_dd (referenced by top-level compiled C/C++ code)
warning: undefined symbol: pow_ri (referenced by top-level compiled C/C++ code)
warning: undefined symbol: r_sign (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_cat (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_cmp (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_copy (referenced by top-level compiled C/C++ code)
warning: undefined symbol: s_wsfe (referenced by top-level compiled C/C++ code)
rsync -r /io/src/../build/release/vosk.js /io/src/../lib/src/vosk-wasm.js
rsync -r vosk.d.ts /io/src/../lib/src/vosk-wasm.d.ts
make: Leaving directory '/io/src'
arbdevml commented 1 year ago

Has anyone successfully implemented this functionality, if yes, could someone share the experience, please? Thank you for your answers.

nshmyrev commented 1 year ago

Has anyone successfully implemented this functionality, if yes, could someone share the experience, please? Thank you for your answers.

It works just fine. You'd better describe what is your trouble in detail.