search

kaldi-asr / kaldi

kaldi-asr/kaldi is the official location of the Kaldi project.
http://kaldi-asr.org
Other
14.24k stars 5.32k forks source link

rnnlm #912

Closed vladbataev closed 8 years ago

vladbataev commented 8 years ago

Hello. I saw rnnlm and faster-rnnlm toolkits which could be used for building language models. But these toolkits only produce neural network weights. How can I use them for creating my own language model?

danpovey commented 8 years ago

RNNLM toolkit produce RNNLM language models, which can be interpreted by those toolkits to produce probabilities. If you want an ARPA-format models, you shouldn't use those toolkits.

Dan

On Fri, Jul 22, 2016 at 9:21 AM, vladbataev notifications@github.com wrote:

Hello. I saw rnnlm and faster-rnnlm toolkits which could be used for building language models. But these toolkits only produce neural network weights. How can I use them for creating my own language model?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/912, or mute the thread https://github.com/notifications/unsubscribe-auth/ADJVu3lcwTiLyx17mxn2PfKP4u6wXfH8ks5qYO4EgaJpZM4JS6hU .

vladbataev commented 8 years ago

So, kaldi provides a conversion of these probabilities into a grammar wfst, doesn't it?

danpovey commented 8 years ago

No, and it's not possible even in principle Dan

On Sun, Jul 24, 2016 at 4:30 PM, Vlad Bataev notifications@github.com wrote:

So, kaldi provides a conversion of these probabilities into a grammar wfst, doesn't it?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/912#issuecomment-234810091, or mute the thread https://github.com/notifications/unsubscribe-auth/ADJVu6ZbeVJ7LvjxlD8XDmsQnooSk3lGks5qY_WEgaJpZM4JS6hU .

vladbataev commented 8 years ago

Then, I understand how rnnlm and n - gram approaches can be combined together or only rnnlm can be used: give n-gram sequence to the neural network and it will produce the probabilities of the next word. After that, it can be used in wsft, but it's constraint of rnnlm, because it doesn't use all history of the previous context. Is it possible to use rnnlm directly?

jtrmal commented 8 years ago

It's probably not possible for any real-size lattice. The current rescoring expands the lattice to get unique 8gram(?, not sure completely of the degree) histories. I think Dan or Guoguo tried higher degree, but it didn't lead to any significant difference. You could, in theory, modify the source code of the rnnlm rescoring to give you higher n-gram contexts. I think it might be even quite easy, provided you are able to read and understand the code. But, In that case, you should be also able to understand why it's not generally reasonable thing to do. Other approach is to generate n-best paths list and rescore the list. There is/was a script for that. I think the finding was the limited n-gram histories worked better than limiting alternatives.

Also, I'm closing this issue. This is not a place to seek this kind of help. y.

On Mon, Jul 25, 2016 at 6:13 AM, Vlad Bataev notifications@github.com wrote:

Then, I understand how rnnlm and n - gram approaches can be combined together or only rnnlm can be used: give n-gram sequence to the neural network and it will produce the probabilities of the next word. After that, it can be used in wsft, but it's constraint of rnnlm, because it doesn't use all history of the previous context. Is it possible to use rnnlm directly?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/912#issuecomment-234915003, or mute the thread https://github.com/notifications/unsubscribe-auth/AKisX0jzq0zo-_JhL5lGb8sHZpODs8IHks5qZIxdgaJpZM4JS6hU .

vladbataev commented 8 years ago

Thanks a lot for your help.