search

renepickhardt / generalized-language-modeling-toolkit

Generalized Language Modeling toolkit
http://glm.rene-pickhardt.de
52 stars 17 forks source link

FalseMaximumLikelihood should work for Marginal #81

Open lschmelzeisen opened 9 years ago

lschmelzeisen commented 9 years ago

You told me FalsemaximumLikelihood-estimation (namely doing c(a b c)/c(a b) instead of c(a b c)/c(a b _)) should work in the marginal case. It doesn't though. We should research why.

renepickhardt commented 9 years ago

we should do a backoff if c(a b) was not seen (if we do proper backoffs this should add to one but it is not MLE)