senarvi
commented
6 years ago In contrast to n-gram models, the RNN probabilities depend on all the words in the partial path from the beginning of the lattice to the target word. So in principle the probabilities are unique for all the words in all the paths through the lattice. To decode a lattice exactly would require evaluating all the paths through the lattice, but that wouldn't be practical. Instead TheanoLM proceeds from the beginning of the lattice to the end, keeping a number of alternative paths, called tokens, in memory at any time.
If the number of tokens get too high, you run out of memory. It also computes a probability for all the tokens that end in the same node in parallel, so you can easily run out of GPU memory unless you set a maximum number for tokens in any node. You can use an approximation that limits how many words of context is considered significant - if the earlier histories of two tokens are identical, the tokens can be recombined by keeping only the best one. You can also specify beam pruning. A more complete description of the pruning options can be found from the documentation: http://theanolm.readthedocs.io/en/latest/applying.html#decoding-word-lattices
I have trained a language model and used it for lattice decoding as part of spelling correction project in my university. The problem is that when I have 5 candidates for like 7 or 8 words in a single lattice, the model is throwing out of memory error, or dumping the core. I'm trying to fix this issue by reducing the number of possible candidates for each spelling error, but this reduces the model performance. It also takes long time to decode a lattice.
Is TheanoLM loading all the possible paths in the lattice in memory and then execute the computations? Dose it use the sequence size in lattice decoding? what is the offset of the sliding window over the words, is it one ? can I change it to be 25 for example? What can I do to be able to decode a lattice without facing this error every time?