Question: how to integrate RNNLM into speech2text ctc with lm decoder?

NVIDIA / OpenSeq2Seq

Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP

https://nvidia.github.io/OpenSeq2Seq

Apache License 2.0

1.54k stars 369 forks source link

Question: how to integrate RNNLM into speech2text ctc with lm decoder? #330

Open d2sys opened 5 years ago

d2sys commented 5 years ago

I think one of the ways is to extend the method _compute_loss inside MultiTaskCTCEntropyLoss class with RNNLM loss

GabrielLin commented 5 years ago

This is a great idea.

xw1324832579 commented 5 years ago

I think one of the ways is to extend the method _compute_loss inside MultiTaskCTCEntropyLoss class with RNNLM loss

Hello, have you integrated RNNLM into speech2text ctc ?After integrated, it will improve the output of the acoustic model or only improve the word-level output text? And ,is the integrating process different from the process of external lm rescore? If you have done this ,can you show the code or please help solve my doubts in detail? Thank you

raikarsagar commented 4 years ago

Any updates regarding this topic? Has anyone implemented this ?