kpu / kenlm

KenLM: Faster and Smaller Language Model Queries
http://kheafield.com/code/kenlm/
Other
2.45k stars 511 forks source link

Segmentation fault after running lmplz #272

Open OOps717 opened 4 years ago

OOps717 commented 4 years ago

Hi. I've just tried to compile the lmplz and faced with the Segmentation fault. Moreover, I was facing with the errors while installation KenLM with Boost version 1.65 which actually i could resolve. I think the problem is in boost, but i would to know if someone think that there is another problem.

Here is the output: ./lmplz --text ../../../ATL/112/vocabulary.txt --arpa words.arpa --o 3 === 1/5 Counting and sorting n-grams ===
Reading /home/oops/ATL/112/vocabulary.txt
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
Segmentation fault (core dumped)

Thank you in advance

kpu commented 4 years ago

This is not good. Can I get a gdb stack trace? Compile with debugging:

cmake -DCMAKE_BUILD_TYPE=Debug ..
make -j 4
cd /whereever/you/were/running #I think that's where you are?
gdb /path/to/kenlm/build/bin/lmplz
run --text ../../../ATL/112/vocabulary.txt --arpa words.arpa --o 3
bt
OOps717 commented 4 years ago

Here is the output:

[Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". === 1/5 Counting and sorting n-grams === Reading /home/oops/ATL/112/vocabulary.txt ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 [New Thread 0x7ffe70baf700 (LWP 12477)] [New Thread 0x7ffe703ae700 (LWP 12478)] [New Thread 0x7ffe6e1ff700 (LWP 12480)]


lmplz: /home/oops/kenlm/util/stream/chain.cc:151: void util::stream::Link::Poison(): Assertion `!poisoned_' failed.

Thread 2 "lmplz" received signal SIGABRT, Aborted. [Switching to Thread 0x7ffe70baf700 (LWP 12477)] __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt

0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51

1 0x00007ffff61c5801 in __GI_abort () at abort.c:79

2 0x00007ffff61b539a in __assert_fail_base (

fmt=0x7ffff633c7d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
assertion=assertion@entry=0x555555767f50 "!poisoned_", 
file=file@entry=0x555555767d50 "/home/oops/kenlm/util/stream/chain.cc", 
line=line@entry=151, 
function=function@entry=0x555555768720 <util::stream::Link::Poison()::__PRETTY_FUNCTION__> "void util::stream::Link::Poison()") at assert.c:92

3 0x00007ffff61b5412 in __GI___assert_fail (

assertion=0x555555767f50 "!poisoned_", 
file=0x555555767d50 "/home/oops/kenlm/util/stream/chain.cc", line=151, 
function=0x555555768720 <util::stream::Link::Poison()::__PRETTY_FUNCTION__> "void util::stream::Link::Poison()") at assert.c:101

4 0x000055555572a5e1 in util::stream::Link::Poison (this=0x7ffe70baea10)

at /home/oops/kenlm/util/stream/chain.cc:151

5 0x00005555556c38ce in lm::builder::(anonymous namespace)::Writer::~Writer (

this=0x7ffe70baea10, __in_chrg=<optimized out>)
at /home/oops/kenlm/lm/builder/corpus_count.cc:86

6 0x00005555556c671a in lm::builder::CorpusCount::RunWithVocab<lm::ngram::GrowableVocab > (this=0x7fffffffd200, position=...,

vocab=...) at /home/oops/kenlm/lm/builder/corpus_count.cc:239

7 0x00005555556c43b5 in lm::builder::CorpusCount::Run (this=0x7fffffffd200,

ganji15 commented 4 years ago

@OOps717 @kpu Hi, I met the same problem. How to solve it, please?

ganji15 commented 4 years ago

@OOps717 @kpu I have solved the problem by updating the boost library to version 1.67.0.

VamsiChagari commented 3 years ago

@OOps717 @kpu I have solved the problem by updating the boost library to version 1.67.0.

It worked for me as well!, thanks.