Cannot load some third-party ARPA-format LMs

emawind84 / mitlm

Automatically exported from code.google.com/p/mitlm

BSD 3-Clause "New" or "Revised" License

0 stars 0 forks source link

Cannot load some third-party ARPA-format LMs #27

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago

What steps will reproduce the problem?

1. Compile trunk (r48)
2. Download an arpa LM e.g. the LM in 
http://www.keithv.com/software/giga/lm_giga_64k_nvp_3gram.zip
3. Run evaluate-ngram which loads the model, e.g.

evaluate-ngram -lm lm_giga_64k_nvp_3gram.arpa -eval-perp data.txt

What is the expected output? What do you see instead?

0.001   Loading LM lm_giga_64k_nvp_3gram.arpa...
Assertion failed: (p >= &line[lineLen]), function LoadLM, file 
src/NgramModel.cpp, line 329.
Abort trap

What version of the product are you using? On what operating system?

r48, MacOSX

Darwin stephen-marquards-macbook-pro.local 10.6.0 Darwin Kernel Version 10.6.0: 
Wed Nov 10 18:13:17 PST 2010; root:xnu-1504.9.26~3/RELEASE_I386 i386

Please provide any additional information below.

Original issue reported on code.google.com by smarqu...@gmail.com on 9 Mar 2011 at 1:01

GoogleCodeExporter commented 8 years ago

Issue 30 describes the same problem.

Original comment by lylepr...@gmail.com on 14 Jul 2011 at 4:49

GoogleCodeExporter commented 8 years ago


interpolate-ngram -lm "your_model.lm, lm_giga_5k_nvp_3gram.arpa" -interpolation 
LI -op msg2.txt -wl Lectures+Textbook.LI.lm
0.000   Loading component LM your_model.lm...
0.000   Loading component LM lm_giga_5k_nvp_3gram.arpa...
interpolate-ngram: src/NgramModel.cpp:329: void 
NgramModel::LoadLM(std::vector<DenseVector<double>, 
std::allocator<DenseVector<double> > >&, std::vector<DenseVector<double>, 
std::allocator<DenseVector<double> > >&, ZFile&): Assertion `p >= 
&line[lineLen]' failed.
Aborted

Original comment by dexteru...@gmail.com on 22 Sep 2011 at 1:06

GoogleCodeExporter commented 8 years ago

Hi,

Was this interpolate-ngram issue resolved ?
I hit the same problem and wonder if the problem
was fixed (or circumvented) to proceed.

Chan

chan@ubuntu:~/Speech/medical/audio/cardiology/LM$ interpolate-ngram -lm 
"hub4.lm" -verbose -interpolation LI -op adapt.txt.for.LM -wl hub4-new.lm
lt-interpolate-ngram: src/NgramModel.cpp:329: void 
NgramModel::LoadLM(std::vector<DenseVector<double> >&, 
std::vector<DenseVector<double> >&, ZFile&): Assertion `p >= &line[lineLen]' 
failed.

Original comment by chan...@gmail.com on 13 Mar 2012 at 1:03

GoogleCodeExporter commented 8 years ago

Hi - I have the exact same issue on centos.  Has anybody found a solution?

Original comment by k...@ksac.com on 28 Aug 2012 at 9:39

GoogleCodeExporter commented 8 years ago

Issue 30 has been merged into this issue.

Original comment by giuliop...@gmail.com on 3 Feb 2013 at 6:25

GoogleCodeExporter commented 8 years ago

[deleted comment]

GoogleCodeExporter commented 8 years ago

Hello, 
I have the same problem withe evaluate :

./evaluate-ngram -lm trial.lm -eval-perp 50.txt

0.001   Loading trial.lm...
lt-evaluate-ngram: src/NgramModel.cpp:330: void 
mitlm::NgramModel::LoadLM(std::vector<mitlm::DenseVector<double> >&, 
std::vector<mitlm::DenseVector<double> >&, mitlm::ZFile&): Assertion `p >= 
&line[lineLen]' failed.
Aborted (core dumped)

Original comment by duygu.al...@gmail.com on 9 Apr 2015 at 12:49