Open lucy9527 opened 8 months ago
csukuangfj
commented
8 months ago 使用最新的zipformer2训练模型，配置如下：
从你给的配置看，你用的是非流式。因为你没有用 --causal=1.
./sherpa-onnx/build-shared/bin/sherpa-onnx
这个只支持流式模型。
你不要把非流式的模型和流式的代码混用了。
针对你这个情况，请使用 sherpa-onnx-offline 等支持非流式模型的 binaries.
lucy9527
commented
8 months ago Sorry to bother you, but three more questions！ 1、sherpa编译报错，部分截取如下
python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `cudaGraphInstantiateWithFlags@libcudart.so.11.0'
anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `ncclCommCount'
anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `cusparseSpSM_destroyDescr@libcusparse.so.11'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `ncclCommInitAll'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `cusparseSpSV_analysis@libcusparse.so.11'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `ncclAllGather'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libc10_cuda.so: undefined reference to `cudaMemPoolTrimTo@libcudart.so.11.0'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `ncclReduce'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so: undefined reference to `cuptiMetricGetIdFromName@libcupti.so.11.7'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libc10_cuda.so: undefined reference to `cudaMallocAsync@libcudart.so.11.0'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `cusparseSpSM_bufferSize@libcusparse.so.11'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so: undefined reference to `cuptiEventGetIdFromName@libcupti.so.11.7'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so: undefined reference to `cuptiActivityEnable@libcupti.so.11.7'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `cusparseSpMatSetAttribute@libcusparse.so.11'
/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so: undefined reference to `cuptiActivityDisable@libcupti.so.11.7'
collect2: error: ld returned 1 exit status
make[2]: *** [sherpa/cpp_api/websocket/CMakeFiles/sherpa-offline-websocket-server.dir/build.make:135: bin/sherpa-offline-websocket-server] Error 1
make[1]: *** [CMakeFiles/Makefile2:641: sherpa/cpp_api/websocket/CMakeFiles/sherpa-offline-websocket-server.dir/all] Error 2
make: *** [Makefile:136: all] Error 2conda list | grep cuda
k2 1.24.3.dev20230718+cuda11.7.torch2.0.1 pypi_0 pypi
k2-sherpa 1.3.dev20230725+cuda11.7.torch2.0.1 pypi_0 pypi
kaldifeat 1.25.0.dev20231022+cuda11.7.torch2.0.1 pypi_0 pypi
nvidia-cuda-cupti-cu11 11.7.101 pypi_0 pypi
nvidia-cuda-nvrtc-cu11 11.7.99 pypi_0 pypi
nvidia-cuda-runtime-cu11 11.7.99 pypi_0 pypi分析：是不是安装都是cuda11.7版本，linux环境目前只有cuda11.1/2版本，是不是得重装 k2 等，再编译
2、sherpa-onnx -> build-static编译报错部分截取如下：
beam_search_scorer.cc:(.text._ZNSt17_Function_handlerIFvPN11onnxruntime7contrib12transformers14BeamHypothesesEEZNS0_10IAllocator13MakeUniquePtrIS3_EESt10unique_ptrIT_St8functionIFvPS9_EEESt10shared_ptrIS6_EmbPNS0_6StreamESA_IFvRSH_RNS0_11synchronize12NotificationEEEEUlS4_E_E10_M_managerERSt9_Any_dataRKSR_St18_Manager_operation[_ZNSt17_Function_handlerIFvPN11onnxruntime7contrib12transformers14BeamHypothesesEEZNS0_10IAllocator13MakeUniquePtrIS3_EESt10unique_ptrIT_St8functionIFvPS9_EEESt10shared_ptrIS6_EmbPNS0_6StreamESA_IFvRSH_RNS0_11synchronize12NotificationEEEEUlS4_E_E10_M_managerERSt9_Any_dataRKSR_St18_Manager_operation]+0xe1): undefined reference to `__libc_single_threaded'
../../_deps/onnxruntime-src/lib/libonnxruntime.a(beam_search_scorer.cc.o): In function `std::_Function_handler<void (onnxruntime::contrib::transformers::HypothesisScore*), std::unique_ptr<onnxruntime::contrib::transformers::HypothesisScore, std::function<void (onnxruntime::contrib::transformers::HypothesisScore*)> > onnxruntime::IAllocator::MakeUniquePtr<onnxruntime::contrib::transformers::HypothesisScore>(std::shared_ptr<onnxruntime::IAllocator>, unsigned long, bool, onnxruntime::Stream*, std::function<void (onnxruntime::Stream&, onnxruntime::synchronize::Notification&)>)::{lambda(onnxruntime::contrib::transformers::HypothesisScore*)#1}>::_M_manager(std::_Any_data&, std::_Function_handler<void (onnxruntime::contrib::transformers::HypothesisScore*), std::unique_ptr<onnxruntime::contrib::transformers::HypothesisScore, std::function<void (onnxruntime::contrib::transformers::HypothesisScore*)> > onnxruntime::IAllocator::MakeUniquePtr<onnxruntime::contrib::transformers::HypothesisScore>(std::shared_ptr<onnxruntime::IAllocator>, unsigned long, bool, onnxruntime::Stream*, std::function<void (onnxruntime::Stream&, onnxruntime::synchronize::Notification&)>)::{lambda(onnxruntime::contrib::transformers::HypothesisScore*)#1}> const&, std::_Manager_operation)':
beam_search_scorer.cc:(.text._ZNSt17_Function_handlerIFvPN11onnxruntime7contrib12transformers15HypothesisScoreEEZNS0_10IAllocator13MakeUniquePtrIS3_EESt10unique_ptrIT_St8functionIFvPS9_EEESt10shared_ptrIS6_EmbPNS0_6StreamESA_IFvRSH_RNS0_11synchronize12NotificationEEEEUlS4_E_E10_M_managerERSt9_Any_dataRKSR_St18_Manager_operation[_ZNSt17_Function_handlerIFvPN11onnxruntime7contrib12transformers15HypothesisScoreEEZNS0_10IAllocator13MakeUniquePtrIS3_EESt10unique_ptrIT_St8functionIFvPS9_EEESt10shared_ptrIS6_EmbPNS0_6StreamESA_IFvRSH_RNS0_11synchronize12NotificationEEEEUlS4_E_E10_M_managerERSt9_Any_dataRKSR_St18_Manager_operation]+0x54): undefined reference to `__libc_single_threaded'
beam_search_scorer.cc:(.text._ZNSt17_Function_handlerIFvPN11onnxruntime7contrib12transformers15HypothesisScoreEEZNS0_10IAllocator13MakeUniquePtrIS3_EESt10unique_ptrIT_St8functionIFvPS9_EEESt10shared_ptrIS6_EmbPNS0_6StreamESA_IFvRSH_RNS0_11synchronize12NotificationEEEEUlS4_E_E10_M_managerERSt9_Any_dataRKSR_St18_Manager_operation[_ZNSt17_Function_handlerIFvPN11onnxruntime7contrib12transformers15HypothesisScoreEEZNS0_10IAllocator13MakeUniquePtrIS3_EESt10unique_ptrIT_St8functionIFvPS9_EEESt10shared_ptrIS6_EmbPNS0_6StreamESA_IFvRSH_RNS0_11synchronize12NotificationEEEEUlS4_E_E10_M_managerERSt9_Any_dataRKSR_St18_Manager_operation]+0xe1): undefined reference to `__libc_single_threaded'
../../_deps/onnxruntime-src/lib/libonnxruntime.a(beam_search_scorer.cc.o): In function `gsl::span<onnxruntime::contrib::transformers::HypothesisScore, 18446744073709551615ul> onnxruntime::rnn::detail::Allocate<onnxruntime::contrib::transformers::HypothesisScore>(std::shared_ptr<onnxruntime::IAllocator>, unsigned long, std::unique_ptr<onnxruntime::contrib::transformers::HypothesisScore, std::function<void (onnxruntime::contrib::transformers::HypothesisScore*)> >&, bool, onnxruntime::contrib::transformers::HypothesisScore)':
beam_search_scorer.cc:(.text._ZN11onnxruntime3rnn6detail8AllocateINS_7contrib12transformers15HypothesisScoreEEEN3gsl4spanIT_Lm18446744073709551615EEESt10shared_ptrINS_10IAllocatorEEmRSt10unique_ptrIS8_St8functionIFvPS8_EEEbS8_[_ZN11onnxruntime3rnn6detail8AllocateINS_7contrib12transformers15HypothesisScoreEEEN3gsl4spanIT_Lm18446744073709551615EEESt10shared_ptrINS_10IAllocatorEEmRSt10unique_ptrIS8_St8functionIFvPS8_EEEbS8_]+0x1b3): undefined reference to `__libc_single_threaded'
../../_deps/onnxruntime-src/lib/libonnxruntime.a(beam_search_scorer.cc.o):beam_search_scorer.cc:(.text._ZN11onnxruntime3rnn6detail8AllocateINS_7contrib12transformers14BeamHypothesesEEEN3gsl4spanIT_Lm18446744073709551615EEESt10shared_ptrINS_10IAllocatorEEmRSt10unique_ptrIS8_St8functionIFvPS8_EEEbS8_[_ZN11onnxruntime3rnn6detail8AllocateINS_7contrib12transformers14BeamHypothesesEEEN3gsl4spanIT_Lm18446744073709551615EEESt10shared_ptrINS_10IAllocatorEEmRSt10unique_ptrIS8_St8functionIFvPS8_EEEbS8_]+0x1b3): more undefined references to `__libc_single_threaded' follow
../../_deps/onnxruntime-src/lib/libonnxruntime.a(utils.cc.o): In function `std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::vector<int, std::allocator<int> > >, true>* std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::vector<int, std::allocator<int> > >, true> > >::_M_allocate_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::vector<int, std::allocator<int> > > const&>(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::vector<int, std::allocator<int> > > const&) [clone .isra.0]':
utils.cc:(.text._ZNSt8__detail16_Hashtable_allocISaINS_10_Hash_nodeISt4pairIKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt6vectorIiSaIiEEELb1EEEEE16_M_allocate_nodeIJRKSD_EEEPSE_DpOT_.isra.0+0x19e): undefined reference to `std::__throw_bad_array_new_length()'
../../_deps/onnxruntime-src/lib/libonnxruntime.a(utils.cc.o): In function `void std::vector<onnxruntime::QDQ::NodeGroup, std::allocator<onnxruntime::QDQ::NodeGroup> >::_M_realloc_insert<onnxruntime::QDQ::NodeGroup const&>(__gnu_cxx::__normal_iterator<onnxruntime::QDQ::NodeGroup*, std::vector<onnxruntime::QDQ::NodeGroup, std::allocator<onnxruntime::QDQ::NodeGroup> > >, onnxruntime::QDQ::NodeGroup const&)':
utils.cc:(.text._ZNSt6vectorIN11onnxruntime3QDQ9NodeGroupESaIS2_EE17_M_realloc_insertIJRKS2_EEEvN9__gnu_cxx17__normal_iteratorIPS2_S4_EEDpOT_[_ZNSt6vectorIN11onnxruntime3QDQ9NodeGroupESaIS2_EE17_M_realloc_insertIJRKS2_EEEvN9__gnu_cxx17__normal_iteratorIPS2_S4_EEDpOT_]+0x366): undefined reference to `std::__throw_bad_array_new_length()'
utils.cc:(.text._ZNSt6vectorIN11onnxruntime3QDQ9NodeGroupESaIS2_EE17_M_realloc_insertIJRKS2_EEEvN9__gnu_cxx17__normal_iteratorIPS2_S4_EEDpOT_[_ZNSt6vectorIN11onnxruntime3QDQ9NodeGroupESaIS2_EE17_M_realloc_insertIJRKS2_EEEvN9__gnu_cxx17__normal_iteratorIPS2_S4_EEDpOT_]+0x375): undefined reference to `std::__throw_bad_array_new_length()'
../../_deps/onnxruntime-src/lib/libonnxruntime.a(utils.cc.o): In function `onnxruntime::QDQ::SelectorManager::InitializeSelectorsMap()':
utils.cc:(.text._ZN11onnxruntime3QDQ15SelectorManager22InitializeSelectorsMapEv+0x5e3): undefined reference to `std::__throw_bad_array_new_length()'
../../_deps/onnxruntime-src/lib/libonnxruntime.a(utils.cc.o): In function `onnxruntime::QDQ::SelectorManager::GetQDQSelections(onnxruntime::GraphViewer const&) const':
utils.cc:(.text._ZNK11onnxruntime3QDQ15SelectorManager16GetQDQSelectionsERKNS_11GraphViewerE+0x92f): undefined reference to `std::__throw_bad_array_new_length()'
../../_deps/onnxruntime-src/lib/libonnxruntime.a(utils.cc.o):utils.cc:(.text._ZNK11onnxruntime3QDQ15SelectorManager16GetQDQSelectionsERKNS_11GraphViewerE+0xc43): more undefined references to `std::__throw_bad_array_new_length()' follow
../../_deps/onnxruntime-src/lib/libonnxruntime.a(regexp.cc.o): In function `re2::ConvertRunesToBytes(bool, int*, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)':
regexp.cc:(.text._ZN3re219ConvertRunesToBytesEbPiiPNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xbe): undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::reserve()'
collect2: error: ld returned 1 exit status
make[2]: *** [sherpa-onnx/csrc/CMakeFiles/sherpa-onnx-offline-tts.dir/build.make:103: bin/sherpa-onnx-offline-tts] Error 1
make[1]: *** [CMakeFiles/Makefile2:1899: sherpa-onnx/csrc/CMakeFiles/sherpa-onnx-offline-tts.dir/all] Error 2
make: *** [Makefile:136: all] Error 23、请问k2的WASR在大数据集和中文效果怎么样啊？
csukuangfj
commented
8 months ago 1、sherpa编译报错，部分截取如下
Please don't use conda, use pip instead.
Please follow
(Never use conda install)
分析：是不是安装都是cuda11.7版本，linux环境目前只有cuda11.1/2版本，是不是得重装 k2 等，再编译
请先看 nvidia-smi 的输出， cuda 版本是多少。（别看 nvcc --version, 是看 nvidia-smi)
2、sherpa-onnx -> build-static编译报错部分截取如下：
Please either use -DBUILD_SHARED_LIBS=ON or use GCC > 9
3、请问k2的WASR在大数据集和中文效果怎么样啊？
这个问题，请去 icefall 的那个 pull request 里面去问下作者。
lucy9527
commented
8 months ago 1、sherpa编译问题：
/usr/bin/ld: warning: libnccl.so.2, needed by **/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so, not found (try using -rpath or -rpath-link)
**/anaconda3/envs/k2/lib/python3.8/site-packages/torch/lib/libtorch_cuda.so: undefined reference to `ncclRecv'
**/anaconda3/envs/k2/lib/python3.8/site-packages/k2/lib/libk2fsa.so: undefined reference to `std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::basic_ostringstream()@GLIBCXX_3.4.26'得安装nccl吗？GLIBCXX_3.4.26？
csukuangfj
commented
8 months ago 1、sherpa编译问题：
请参考上面的链接：
别用 conda install 去安装 torch 和 cudatoolkit. 再重复一遍，不要用 conda install.
csukuangfj
commented
8 months ago 安装新一代 kaldi 里面的项目，请完全按照我们提供的文档来。
如果你能自己解决问题，可以自行发挥。不然，请按照我们提供的文档，谢谢合作。
lucy9527
commented
8 months ago 好的，感谢！
使用最新的zipformer2训练模型，配置如下：
export_onnx.py导出模型，配置如下：
./sherpa-onnx/build-shared/bin/sherpa-onnx \ --tokens=lang_char/tokens.txt \ --encoder=$par/encoder-epoch-30-avg-2.onnx \ --decoder=$par/decoder-epoch-30-avg-2.onnx \ --joiner=$par/joiner-epoch-30-avg-2.onnx \ ./demo.wav 运行如下：