search

kpu / kenlm

KenLM: Faster and Smaller Language Model Queries
http://kheafield.com/code/kenlm/
Other
2.5k stars 513 forks source link

Use lmplz prune a .klm model #241

Closed Chen1399 closed 5 years ago

Chen1399 commented 5 years ago

Hi, I have a question. How to prune a .klm model. Can I use lmplz do it? bin/lmplz -o5 -prune 0 1 1 2 2 ... ? Thank you.

kpu commented 5 years ago

lmplz can prune a model built from scratch as you suggest. In a trained model there isn't sufficient information to perform the same pruning.

Chen1399 commented 5 years ago

Thanks a lot.

lmplz can prune a model built from scratch as you suggest. In a trained model there isn't sufficient information to perform the same pruning.

Thanks a lot.

udaram commented 4 years ago

./kenlm/bin/lmplz -o3 --prune 0 1 1 <data.txt >data.arpa output :- unrecognized option '--prune'

without prune command is working fine for me

@kpu --prune is not working for me in Kenelm toolkit. could you please resolve my this issue and please also tell me what is the meaning of numbers after --prune option what will happen if we replace 0 by 1 or 1 by 2. @chen1399 if you have resolved your issue please help

kpu commented 4 years ago

@udaram You have an old version of the software. Get it from this repo.