getting error on interpolate: Streams were not the same size during merging

kpu / kenlm

KenLM: Faster and Smaller Language Model Queries

http://kheafield.com/code/kenlm/

Other

2.5k stars 513 forks source link

getting error on interpolate: Streams were not the same size during merging #259

Closed besimali closed 4 years ago

besimali commented 4 years ago

Hi,

I'm getting an error when running interpolate with weights: interpolate -m lm_{domain,general} -w 0.2 0.8 > lm.arpa

The error is:

backoff_reunification.cc:33 in void lm::interpolate::{anonymous}::MergeWorker::Run(const util::stream::ChainPosition&) threw util::Exception because `prob_input || boff_input'.
Streams were not the same size during merging
Aborted (core dumped)

besimali commented 4 years ago

Closing this issue as it seems to be connected to #177 because when I interpolate the lm_domain and lm_general without --limit_vocab_file it runs fine.