Closed Sushil-Thapa closed 5 years ago
the relevant part of the code can be found here: applyLM(...)
Further, you need to create a lookup-table containing the n-gram (e.g. n=3) probabilities in the class LanguageModel, similar to initCharBigrams(...).
Thank you for your reply. Yes, I've created the lookup for trigrams in LanguageModel.
Does it makes sense to get the second char(prior) for trigram by
classes[parentBeam.labeling[-2] if parentBeam.labeling else classes.index(' ')]
just like *labeling[-1]*
for first char?
[third char now will be conditioned on first and second char]
What I used for the case n=2 was a trick to handle the case that there is only one character in the beam-text. Then I just say: well, let's assume there is a whitespace as the first character (which somehow makes sense for written text).
You could also implement a fall back; let's say you use n-gram with n=3:
I would suggest reading the relevant chapters (especially "Language Modeling with N- grams") in the book "Speech and Language Processing" from Jurafsky. There you can multiple useful methods and algorithms which can be implemented into the language model of the decoder.
I wanted to experiment with the LMs other than Bigram. Any suggestion on how to approach extending current codebase? or the probability of the last character conditioned on all previous character.