kpu / kenlm

KenLM: Faster and Smaller Language Model Queries
http://kheafield.com/code/kenlm/
Other
2.5k stars 513 forks source link

kenlm for 10 gram #358

Closed Ani2019 closed 2 years ago

Ani2019 commented 2 years ago

Hi I need to compile kenlm to train a 10-gram language model. I add the following ifndef statement to kenlm/utils/have.hh, change the setup.py, but I have the following error:

FormatLoadException because `counts.size() > 6'. This model has order 10 but KenLM was compiled to support up to 6. If your build system supports changing KENLM_MAX_ORDER, change it there and recompile. With cmake: cmake -DKENLM_MAX_ORDER=10 .. With Moses: bjam --max-kenlm-order=10 -a Otherwise, edit lm/max_order.hh. Byte: 146 ERROR Training KenLM was not successful!