search

mozilla / DeepSpeech

DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.
Mozilla Public License 2.0
25.03k stars 3.94k forks source link

Use CTC beam search decoder with subword encoding. #3750

Open DomainFlag opened 1 year ago

DomainFlag commented 1 year ago

I'm using the scorer generator provided generate_scorer_package. I'm also using (e.g., SentencePiece) to build a unigram language model, where the decoder predicts the size of the language model. How can I adapt the scorer such that it supports sub-word units? Will scorer work if filling the alphabet file with the sub-word units? Or shall I rely on some tricks like encoding the unigram language model using an ASCII table and re-encoding the corpus and use the alphabet based on the previous encoding mapping? Thank you.

huks0 commented 1 month ago

Have you ever solved this? Is there a way to use subword encoding?