search

sandrohanea / whisper.net

Whisper.net. Speech to text made simple using Whisper Models
MIT License
506 stars 77 forks source link

How do I add known words to be in the audio? #141

Closed abbotadmin closed 7 months ago

abbotadmin commented 7 months ago

I will be processing a bunch of text that has known weird names like "Mr Jommbla" or "El Paso De Marko". Can I supply a list of phrases prior to processing to improve the likelihood of correctly capturing what is said?

sandrohanea commented 7 months ago

What you want is probably fine tuning and unfortunately there is no easy way of doing it but luckly it is possible.

Here it is described how to convert fine-tuned models back to ggml + a link to blog about creating finetuned models: https://github.com/ggerganov/whisper.cpp/blob/master/models/README.md#fine-tuned-models

You can also play with WithPrompt method to try some prompts to indicate that those are valid outputs, but I don't think you will be able to archive more only with this: https://github.com/sandrohanea/whisper.net/blob/main/Whisper.net/WhisperProcessorBuilder.cs#L286