search

mozilla / DeepSpeech

DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.
Mozilla Public License 2.0
25.36k stars 3.97k forks source link

Replace KenLM Language Model with TF Language Model #669

Open kdavis-mozilla opened 7 years ago

rajateku6 commented 6 years ago

@kdavis-mozilla can we have our own language model using the existing acoustic model. If so please can you give some glimpse of it.

kdavis-mozilla commented 6 years ago

@rajateku6 This is suggesting a specific enhancement to the code. If you wish to train another KenLM language model please add your question to the Discourse Forum.

ksteimel commented 5 years ago

I would like to work on making the language model interface more general. Do you think this would be a good approach:

Make Scorer a generic class that subtypes of scorers inherit from (e.g. KenlmScorer). Each subtype of scorer would have to implement get_log_prob() in their own way. This way, someone could write their own scorer using tensorflow much more easily. @ftyers

reuben commented 5 years ago

Thanks for the interest, that'd be a great contribution! The approach sounds good, although I'm not entirely sure get_lob_prob is the only API we'd have to surface from the Scorer. There's also the question of different scorers having different initialization and configuration parameters, and how to expose that in the main DeepSpeech API. But that's a worry for the future, just having an initial abstraction that can be used to experiment with new implementations would be really great to have.

kdavis-mozilla commented 4 years ago

Adding this to the 1.0.0 project. This may get implemented there, and it may not dependent upon the LM work done there

zaptrem commented 4 years ago

Was this plan delayed or abandoned?

rhamnett commented 4 years ago

FWIW think this would be an excellent addition, TF or torch model, maybe leverage transformer library or something similar

zaptrem commented 4 years ago

FWIW think this would be an excellent addition, TF or torch model, maybe leverage transformer library or something similar

I agree. The current model seems to underperform the competition In the real world while requiring a much larger filesize. In the last two years transformers have revolutionized the LM landscape. Architectures like BERT/DistilBERT/XLM also open interesting opportunities for improving a transcribed word based on context heard afterwards.

tohalb commented 3 years ago

It looks as though incorporating transformers as a language model into DeepSpeech has been on people's Wishlist for a while. Does anyone know if any progress has been made or if anyone has managed this? I've looked but can't find anything - not sure if I'm missing something.