Possible to add both Word LM AND Character LM?

[thetruejacob]

Thank you for the consistent updates to this project. I'm currently using this in conjunction with SimpleHTR, and it's working wonders so far. I'm wondering if it's possible to combine knowledge both on the character level as well as the word level, or is it only possible to do word-level modeling?

And how exactly should the language model be prescribed? Can I use NLTK to build the LM, or what is the recommended method for creating it?

[githubharald]

only a word-level language model is implemented. You would have to change the code to handle char-level language models. Not sure it combining both types makes sense ... have a look into language model literature what they write about this. At least I never heard of something like that.

[thetruejacob]

I've checked your other repo, named CTCDecoder. That module seems to allow character-level language models, but can I use that functionality with SimpleHTR?

[weinman]

A character level language model is not unreasonable (particularly if you think of it as an old school back-off model), though if you're presumably training discriminative recognition model, you're doing a bit of ex situ / ex post facto tinkering with the learned biases.

Anyway, you could implement that quite easily by just adding a differential bias to each character before putting it through the decoder.

For example, if the raw character-level scores are TxBx(C+1), you'd add a (C+1) vector of the character-specific biases to the score tensor (using the magic of broadcasting) before re-normalizing and putting it through the decoder. (If you don't have raw logits, you could multiply by normalized character LM scores instead of adding the biases.)

[githubharald]

I've checked your other repo, named CTCDecoder. That module seems to allow character-level language models, but can I use that functionality with SimpleHTR?

yes, but the decoders implemented in that repo do not implement batch-mode. So, you have to go through all batch elements yourself and apply the decoder to each batch element separately (see README in the CTCDecoder repo).