search

amit-bhavsar / mitlm

Automatically exported from code.google.com/p/mitlm
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

Error loading arpa format lm #39

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Try to load arpa lm using evaluate-lm

I have tried with ngrams estimated using both mitlm as well as other tool sets.

What is the expected output? What do you see instead?

0.000   Loading LM exp/arpa/imd/imd.p7E-8.arpa...
terminate called after throwing an instance of 'std::invalid_argument'
  what():  Unexpected file format.

What version of the product are you using? On what operating system?

Used both latest SVN trunk and 0.4.1 on Ubuntu Linux

Please provide any additional information below.

The problem seems to be on line 293 of NgramModel.cpp

"if (sscanf(line, "\\%u-ngrams:", &i) != 1 || i != o) {"

On line 382 we see the problem, when writing out the n-gram:
"fprintf(lmFile, "\n\\%lu-grams:\n", (unsigned long)o);"

Thus the fix is fairly straight forward, just change line 293 to:
"if (sscanf(line, "\\%u-grams:", &i) != 1 || i != o) {"

Original issue reported on code.google.com by john.di...@gmail.com on 18 May 2014 at 9:52

GoogleCodeExporter commented 9 years ago 
Thank you for reporting the issue and providing the fix.
It has been applied in revision r104.

Bests,
Giulio.

Original comment by giuliop...@gmail.com on 2 Jun 2014 at 11:19