Positive log probability... in the model when creating model with --collapse_values

kpu / kenlm

KenLM: Faster and Smaller Language Model Queries

http://kheafield.com/code/kenlm/

Other

2.46k stars 514 forks source link

Positive log probability... in the model when creating model with --collapse_values #400

Open huu4ontocord opened 1 year ago

huu4ontocord commented 1 year ago

Hi, I see that the collapse values can be positive, and when loading the model using kenlm.Model, I get a complaint that there are positive log probs. Is there a way to fix this problem?