search

kpu / kenlm

KenLM: Faster and Smaller Language Model Queries
http://kheafield.com/code/kenlm/
Other
2.5k stars 513 forks source link

core dump occurred when load LM model #328

Closed Gavin90s closed 3 years ago

Gavin90s commented 3 years ago

Core was generated by `python examples/speech_recognition/infer.py manifest/finetune/ --task audio_pre'. Program terminated with signal 11, Segmentation fault.

0 _M_data (this=0x2ac160808) at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/build/build-cc-gcc-final/x86_64-conda_cos6-linux-gnu/libstdc++-v3/include/bits/basic_string.h:229

229 /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/build/build-cc-gcc-final/x86_64-conda_cos6-linux-gnu/libstdc++-v3/include/bits/basic_string.h: No such file or directory. Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.1.alios7.x86_64 glibc-2.17-222.alios7.1.x86_64 libuuid-2.23.2-43.alios7.2.x86_64 (gdb) bt

0 _M_data (this=0x2ac160808) at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/build/build-cc-gcc-final/x86_64-conda_cos6-linux-gnu/libstdc++-v3/include/bits/basic_string.h:229

1 _M_is_local (this=0x2ac160808) at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/build/build-cc-gcc-final/x86_64-conda_cos6-linux-gnu/libstdc++-v3/include/bits/basic_string.h:222

2 std::__cxx11::basic_string<char, std::char_traits, std::allocator >::_M_dispose (this=0x2ac160808)

at /home/nwani/m3/conda-bld/compilers_linux-64_1560109574129/work/.build/x86_64-conda_cos6-linux-gnu/build/build-cc-gcc-final/x86_64-conda_cos6-linux-gnu/libstdc++-v3/include/bits/basic_string.h:231

3 0x00007f3f79c44825 in std::_Destroy<std::cxx11::basic_string<char, std::char_traits, std::allocator > > (pointer=0x2ac160808) at /usr/local/include/c++/5.2.0/bits/stl_construct.h:93

4 0x00007f3f79c44583 in std::_Destroy_aux::destroy<std::cxx11::basic_string<char, std::char_traits, std::allocator >*> (first=0x2ac160808, last=0x0) at /usr/local/include/c++/5.2.0/bits/stl_construct.h:103

5 0x00007f3f79c44220 in std::_Destroy<std::cxx11::basic_string<char, std::char_traits, std::allocator >*> (first=0x2ac160808, __last=0x0) at /usr/local/include/c++/5.2.0/bits/stl_construct.h:126

6 0x00007f3f79c43d73 in std::_Destroy<std::cxx11::basic_string<char, std::char_traits, std::allocator >*, std::cxx11::basic_string<char, std::char_traits, std::allocator > > (first=0x2ac160808, last=0x0)

at /usr/local/include/c++/5.2.0/bits/stl_construct.h:151

7 0x00007f3f79c438ad in std::vector<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::cxx11::basic_string<char, std::char_traits, std::allocator > > >::~vector (

this=0x7ffcd25ff1c8, __in_chrg=<optimized out>) at /usr/local/include/c++/5.2.0/bits/stl_vector.h:424

8 0x00007f3f79c5753c in lm::ngram::Config::~Config (this=0x7ffcd25ff150, __in_chrg=) at /usr/local/include/kenlm/lm/config.hh:19

9 0x00007f3f79c56bec in fl::lib::text::KenLM::KenLM (this=0x7f401cd0ed70, path=..., usrTknDict=...) at /disk4/zhuozhu.zz/flashlight/flashlight/lib/text/decoder/lm/KenLM.cpp:23

10 0x00007f3facb487cc in pybind11::detail::initimpl::construct_or_initialize<fl::lib::text::KenLM, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&, 0> ()

at /disk4/zhuozhu.zz/flashlight/bindings/python/build/temp.linux-x86_64-3.6/pybind11/src/pybind11/include/pybind11/detail/init.h:61

11 0x00007f3facb36e9c in void pybind11::detail::initimpl::constructor<std::cxx11::basic_string<char, std::chartraits, std::allocator > const&, fl::lib::text::Dictionary const&>::execute<pybind11::class<fl::lib::text::KenLM, std::sharedptr, fl::lib::text::LM>, pybind11::arg, pybind11::arg, 0>(pybind11::class<fl::lib::text::KenLM, std::shared_ptr, fl::lib::text::LM>&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&)#1}::operator()(pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&) const (closure=0x7f401d4ce468, v_h=..., args#0=..., args#1=...)

at /disk4/zhuozhu.zz/flashlight/bindings/python/build/temp.linux-x86_64-3.6/pybind11/src/pybind11/include/pybind11/detail/init.h:174

12 0x00007f3facb7e3c5 in pybind11::detail::argument_loader<pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&>::call_impl<void, void pybind11::detail::initimpl::constructor<std::cxx11::basic_string<char, std::chartraits, std::allocator > const&, fl::lib::text::Dictionary const&>::execute<pybind11::class<fl::lib::text::KenLM, std::sharedptr, fl::lib::text::LM>, pybind11::arg, pybind11::arg, 0>(pybind11::class<fl::lib::text::KenLM, std::shared_ptr, fl::lib::text::LM>&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&)#1}&, 0ul, 1ul, 2ul, pybind11::detail::void_type>(void pybind11::detail::initimpl::constructor<std::cxx11::basic_string<char, std::chartraits, std::allocator > const&, fl::lib::text::Dictionary const&>::execute<pybind11::class<fl::lib::text::KenLM, std::sharedptr, fl::lib::text::LM>, pybind11::arg, pybind11::arg, 0>(pybind11::class<fl::lib::text::KenLM, std::shared_ptr, fl::lib::text::LM>&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::value_and_holder&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&)#1}&, std::integer_sequence<unsigned long, 0ul, 1ul, 2ul>, pybind11::detail::void_type&&) (this=0x7ffcd25ff3f0, f=...)

at /disk4/zhuozhu.zz/flashlight/bindings/python/build/temp.linux-x86_64-3.6/pybind11/src/pybind11/include/pybind11/cast.h:1874

13 0x00007f3facb7856b in _ZNO8pybind116detail15argument_loaderIJRNS0_16value_and_holderERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKN2fl3lib4text10DictionaryEEE4callIvNS0_9void_typeERZNS0_8initimpl11constructorIJSB_SH_EE7executeINS_6class_INSE_5KenLMEJSt10shared_ptrISQ_ENSE_2LMEEEEJNS_3argESV_ELi0EEEvRT_DpRKT0_EUlS3_SB_SH_E_EENSt9enable_ifIXsrSt7is_voidISW_E5valueESKE4typeEOT1 (this=0x7ffcd25ff3f0, f=...)

at /disk4/zhuozhu.zz/flashlight/bindings/python/build/temp.linux-x86_64-3.6/pybind11/src/pybind11/include/pybind11/cast.h:1856

14 0x00007f3facb6ad1e in void pybind11::cpp_function::initialize<void pybind11::detail::initimpl::constructor<std::cxx11::basic_string<char, std::chartraits, std::allocator > const&, fl::lib::text::Dictionary const&>::execute<pybind11::class<fl::lib::text::KenLM, std::sharedptr, fl::lib::text::LM>, pybind11::arg, pybind11::arg, 0>(pybind11::class<fl::lib::text::KenLM, std::shared_ptr, fl::lib::text::LM>&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&)#1}, void, pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::detail::is_new_style_constructor, pybind11::arg, pybind11::arg>(void pybind11::detail::initimpl::constructor<std::cxx11::basic_string<char, std::chartraits, std::allocator > const&, fl::lib::text::Dictionary const&>::execute<pybind11::class<fl::lib::text::KenLM, std::sharedptr, fl::lib::text::LM>, pybind11::arg, pybind11::arg, 0>(pybind11::class<fl::lib::text::KenLM, std::shared_ptr, fl::lib::text::LM>&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&)#1}&&, void (*)(pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::detail::is_new_style_constructor const&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::function_call&)#3}::operator()(pybind11::detail::function_call) const (__closure=0x0, call=...)

at /disk4/zhuozhu.zz/flashlight/bindings/python/build/temp.linux-x86_64-3.6/pybind11/src/pybind11/include/pybind11/pybind11.h:154

15 0x00007f3facb6af93 in void pybind11::cpp_function::initialize<void pybind11::detail::initimpl::constructor<std::cxx11::basic_string<char, std::chartraits, std::allocator > const&, fl::lib::text::Dictionary const&>::execute<pybind11::class<fl::lib::text::KenLM, std::sharedptr, fl::lib::text::LM>, pybind11::arg, pybind11::arg, 0>(pybind11::class<fl::lib::text::KenLM, std::shared_ptr, fl::lib::text::LM>&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&)#1}, void, pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::detail::is_new_style_constructor, pybind11::arg, pybind11::arg>(void pybind11::detail::initimpl::constructor<std::cxx11::basic_string<char, std::chartraits, std::allocator > const&, fl::lib::text::Dictionary const&>::execute<pybind11::class<fl::lib::text::KenLM, std::sharedptr, fl::lib::text::LM>, pybind11::arg, pybind11::arg, 0>(pybind11::class<fl::lib::text::KenLM, std::shared_ptr, fl::lib::text::LM>&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&)#1}&&, void (*)(pybind11::detail::value_and_holder&, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&, fl::lib::text::Dictionary const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::detail::is_new_style_constructor const&, pybind11::arg const&, pybind11::arg const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call) ()

at /disk4/zhuozhu.zz/flashlight/bindings/python/build/temp.linux-x86_64-3.6/pybind11/src/pybind11/include/pybind11/pybind11.h:132

16 0x00007f3facb1f828 in pybind11::cpp_function::dispatcher (self=0x7f3f22fd6ab0, args_in=0x7f3fad082c60, kwargs_in=0x0)

at /disk4/zhuozhu.zz/flashlight/bindings/python/build/temp.linux-x86_64-3.6/pybind11/src/pybind11/include/pybind11/pybind11.h:627

17 0x00007f401a7e7a14 in _PyCFunction_FastCallDict () at /tmp/build/80754af9/python_1599604603603/work/Objects/methodobject.c:231

18 0x00007f401a7e7e2f in _PyObject_FastCallDict () at /tmp/build/80754af9/python_1599604603603/work/Objects/abstract.c:2313

19 0x00007f401a7ec873 in _PyObject_Call_Prepend () at /tmp/build/80754af9/python_1599604603603/work/Objects/abstract.c:2373

20 0x00007f401a7e781e in PyObject_Call () at /tmp/build/80754af9/python_1599604603603/work/Objects/abstract.c:2261

21 0x00007f401a84088b in slot_tp_init () at /tmp/build/80754af9/python_1599604603603/work/Objects/typeobject.c:6420

22 0x00007f401a86fd97 in type_call () at /tmp/build/80754af9/python_1599604603603/work/Objects/typeobject.c:915

23 0x00007f401a7e7bfb in _PyObject_FastCallDict () at /tmp/build/80754af9/python_1599604603603/work/Objects/abstract.c:2331

24 0x00007f401a86fbae in call_function () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4875

---Type to continue, or q to quit---

25 0x00007f401a89225a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:3335

26 0x00007f401a869166 in _PyEval_EvalCodeWithName () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4166

27 0x00007f401a86a32c in _PyFunction_FastCallDict () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:5084

28 0x00007f401a7e7ddf in _PyObject_FastCallDict () at /tmp/build/80754af9/python_1599604603603/work/Objects/abstract.c:2310

29 0x00007f401a7ec873 in _PyObject_Call_Prepend () at /tmp/build/80754af9/python_1599604603603/work/Objects/abstract.c:2373

30 0x00007f401a7e781e in PyObject_Call () at /tmp/build/80754af9/python_1599604603603/work/Objects/abstract.c:2261

31 0x00007f401a84088b in slot_tp_init () at /tmp/build/80754af9/python_1599604603603/work/Objects/typeobject.c:6420

32 0x00007f401a86fd97 in type_call () at /tmp/build/80754af9/python_1599604603603/work/Objects/typeobject.c:915

33 0x00007f401a7e7bfb in _PyObject_FastCallDict () at /tmp/build/80754af9/python_1599604603603/work/Objects/abstract.c:2331

34 0x00007f401a86fbae in call_function () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4875

35 0x00007f401a89225a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:3335

36 0x00007f401a869166 in _PyEval_EvalCodeWithName () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4166

37 0x00007f401a869e51 in fast_function () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4992

38 0x00007f401a86fb35 in call_function () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4872

39 0x00007f401a89225a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:3335

40 0x00007f401a8692ce in _PyEval_EvalCodeWithName () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4166

41 0x00007f401a869e51 in fast_function () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4992

42 0x00007f401a86fb35 in call_function () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4872

43 0x00007f401a89225a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:3335

44 0x00007f401a869c1b in _PyFunction_FastCall (globals=, nargs=0, args=, co=) at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4933

45 fast_function () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4968

46 0x00007f401a86fb35 in call_function () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4872

47 0x00007f401a89225a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:3335

48 0x00007f401a86a969 in _PyEval_EvalCodeWithName (qualname=0x0, name=0x0, closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwstep=2, kwcount=, kwargs=0x0, kwnames=0x0, argcount=0, args=0x0, locals=0x7f401a62c1b0,

globals=0x7f401a62c1b0, _co=0x7f4012fd8390) at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4166

49 PyEval_EvalCodeEx () at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:4187

50 0x00007f401a86b70c in PyEval_EvalCode (co=co@entry=0x7f4012fd8390, globals=globals@entry=0x7f401a62c1b0, locals=locals@entry=0x7f401a62c1b0) at /tmp/build/80754af9/python_1599604603603/work/Python/ceval.c:731

51 0x00007f401a8eb574 in run_mod () at /tmp/build/80754af9/python_1599604603603/work/Python/pythonrun.c:1025

52 0x00007f401a8eb971 in PyRun_FileExFlags () at /tmp/build/80754af9/python_1599604603603/work/Python/pythonrun.c:978

53 0x00007f401a8ebb73 in PyRun_SimpleFileExFlags () at /tmp/build/80754af9/python_1599604603603/work/Python/pythonrun.c:419

54 0x00007f401a8ebc7d in PyRun_AnyFileExFlags () at /tmp/build/80754af9/python_1599604603603/work/Python/pythonrun.c:81

55 0x00007f401a8ef663 in run_file (p_cf=0x7ffcd2600d2c, filename=0x7f401ac41440 L"examples/speech_recognition/infer.py", fp=0x7f401ac79b10) at /tmp/build/80754af9/python_1599604603603/work/Modules/main.c:340

56 Py_Main () at /tmp/build/80754af9/python_1599604603603/work/Modules/main.c:811

57 0x00007f401a7b943e in main () at /tmp/build/80754af9/python_1599604603603/work/Programs/python.c:69

58 0x00007f4019eea445 in __libc_start_main () from /lib64/libc.so.6

59 0x00007f401a898d0b in _start () at ../sysdeps/x86_64/elf/start.S:103

kenlm load lm core dump centos 7 python 3.6

LM is ok, when use query binary of kenlm.

Is there anyone can help me ? thanks in advance.

cocoking99 commented 2 years ago

@Gavin90s I have met the same problem, may I ask whether you have solved this problem?

kamirdin commented 2 years ago

@Gavin90s @cocoking99 I have met the same problem, may I ask whether you have solved this problem? :cry:

cocoking99 commented 2 years ago

my problem is in aarch64, and I fixed this problem by adding "set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=0")" to the CMakeLists.txt