search

flashlight / wav2letter

Facebook AI Research's Automatic Speech Recognition Toolkit
https://github.com/facebookresearch/wav2letter/wiki
Other
6.39k stars 1.01k forks source link

tests fail (error: 'afSetMemStepSize' is not a member of 'fl') while building Dockerfile-CPU #739

Closed bohdantrotsenko closed 4 years ago

bohdantrotsenko commented 4 years ago

On AWS I launched Deep Learning AMI (Ubuntu 18.04) Version 30.0 - ami-049fb1ea198d189d7 on a t3a.2xlarge instance.

As I cloned the project (git clone https://github.com/facebookresearch/wav2letter) the directory was at commit 18a1dfe4b22e875e5fd1f875f7bf9ec88a5616cd

I failed to build docker build -f Dockerfile-CPU . -t wavcpu, with the two errors:


[ 86%] Linking CXX executable ListFileDatasetTest                                                                                                              
Scanning dependencies of target decode_len_lpm                                                                                                                  
[ 86%] Building CXX object recipes/models/local_prior_match/CMakeFiles/decode_len_lpm.dir/Decode_length_lpm.cpp.o                                                
/root/wav2letter/recipes/models/local_prior_match/Decode_length_lpm.cpp: In function 'int main(int, char**)':                                              
/root/wav2letter/recipes/models/local_prior_match/Decode_length_lpm.cpp:66:7: error: 'afSetMemStepSize' is not a member of 'fl'                                                                                  
   fl::afSetMemStepSize(FLAGS_memstepsize);                                                                                                                                                                              ^~~~~~~~~~~~~~~~                

...
[ 86%] Building CXX object recipes/models/local_prior_match/CMakeFiles/wav2letter++_lpm_oss.dir/__/__/__/src/libraries/lm/KenLM.cpp.o             
[ 86%] Built target CeplifterTest                                                                                                            
[ 86%] Building CXX object recipes/models/local_prior_match/CMakeFiles/wav2letter++_lpm_oss.dir/__/__/__/src/libraries/lm/ConvLM.cpp.o             
[ 86%] Building CXX object recipes/models/local_prior_match/CMakeFiles/wav2letter++_lpm_oss.dir/__/__/__/src/libraries/lm/ZeroLM.cpp.o        
[ 86%] Building CXX object recipes/models/local_prior_match/CMakeFiles/wav2letter++_lpm_oss.dir/__/__/__/src/libraries/feature/Ceplifter.cpp.o        
recipes/models/local_prior_match/CMakeFiles/decode_len_lpm.dir/build.make:62: recipe for target 'recipes/models/local_prior_match/CMakeFiles/decode_len_lpm.dir/Decode_length_lpm.cpp.o' failed
make[2]: *** [recipes/models/local_prior_match/CMakeFiles/decode_len_lpm.dir/Decode_length_lpm.cpp.o] Error 1                                           
CMakeFiles/Makefile2:1529: recipe for target 'recipes/models/local_prior_match/CMakeFiles/decode_len_lpm.dir/all' failed                           
make[1]: *** [recipes/models/local_prior_match/CMakeFiles/decode_len_lpm.dir/all] Error 2                                                        
make[1]: *** Waiting for unfinished jobs....                                                                                                                
[ 86%] Building CXX object recipes/models/local_prior_match/CMakeFiles/wav2letter++_lpm_oss.dir/__/__/__/src/libraries/feature/Dct.cpp.o                  
[ 87%] Building CXX object recipes/models/local_prior_match/CMakeFiles/wav2letter++_lpm_oss.dir/__/__/__/src/libraries/feature/Derivatives.cpp.o 

Here is the end of the output (probably not informative):
[ 93%] Building CXX object recipes/models/utilities/convlm_serializer/CMakeFiles/SerializeConvLM.dir/__/__/__/__/src/runtime/Serial.cpp.o
[ 93%] Building CXX object recipes/models/utilities/convlm_serializer/CMakeFiles/SerializeConvLM.dir/__/__/__/__/src/runtime/SpeechStatMeter.cpp.o
[ 93%] Building CXX object recipes/models/utilities/convlm_serializer/CMakeFiles/SerializeConvLM.dir/__/__/__/__/src/runtime/Distributed.cpp.o
[ 93%] Building CXX object recipes/models/utilities/convlm_serializer/CMakeFiles/SerializeConvLM.dir/__/__/__/__/src/runtime/Optimizer.cpp.o
[ 93%] Building CXX object recipes/models/utilities/convlm_serializer/CMakeFiles/SerializeConvLM.dir/__/__/__/__/src/runtime/Helpers.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/data/Sound.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/data/Utils.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/data/W2lDataset.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/data/W2lBlobsDataset.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/data/W2lListFilesDataset.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/module/ConvLmModule.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/module/SpecAugment.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/module/TDSBlock.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/module/W2lModule.cpp.o
[ 93%] Linking CXX executable SerializeConvLM
[ 93%] Built target SerializeConvLM
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/runtime/Data.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/runtime/Logger.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/runtime/Serial.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/runtime/SpeechStatMeter.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/runtime/Distributed.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/runtime/Optimizer.cpp.o
[ 93%] Building CXX object src/tests/CMakeFiles/DerivativesTest.dir/__/runtime/Helpers.cpp.o
[ 94%] Linking CXX executable DerivativesTest
[ 94%] Built target DerivativesTest
make: *** [all] Error 2
Makefile:94: recipe for target 'all' failed
The command '/bin/sh -c export MKLROOT=/opt/intel/mkl &&     cd /root && git clone --recursive https://github.com/facebookresearch/flashlight.git &&     cd flashlight && mkdir -p build && cd build &&     cmake
.. -DCMAKE_BUILD_TYPE=Release -DFLASHLIGHT_BACKEND=CPU &&     make -j$(nproc) && make install &&     export KENLM_ROOT_DIR=/root/kenlm &&     cd /root/wav2letter && mkdir -p build &&     cd build && cmake .. -DCMAKE_BUILD_TYPE=Release -DW2L_LIBRARIES_USE_CUDA=OFF &&     make -j$(nproc)' returned a non-zero code: 2```
bohdantrotsenko commented 4 years ago

I also tried commit 9e2c69f1962900c5e9ef723ab14047962b4458e2 and modified Dockerfile-CPU:

cd /root && git clone --recursive https://github.com/facebookresearch/flashlight.git && \
cd flashlight && git checkout 16d0bf0bf1dfb930b7bb0415260f9e7914f7818c && mkdir -p build && cd build && \

as suggested by @an918tw 3 days ago

But still received a different (and bigger) set of errors (attached) dockerbuild.txt

tlikhomanenko commented 4 years ago

Could you try wav2letter/wav2letter:cpu-9e2c69 docker (which was built on that commit)?

Regarding your build - can you add ls -lah after cd /root/wav2letter and check in the log that Test.cpp will be in the list. So far seems that there is no copy of part of files in the wav2letter folder in the docker. Also problem could be in usage wav2letter/wav2letter:cpu-base-latest so you can try to switch cpu-base-26c69be which we used for these commits of w2l and flashlight.

bohdantrotsenko commented 4 years ago

While I repeated experiments last week on hash 18a1dfe4 and still had problems, Right now - hash d6f0fac8 - is built sucessfully :)