Closed Kentoseth closed 3 months ago
Thanks for the question.
Indeed, "Punctuation restoration" function is definitely not well explained in the app - sorry.
"Punctuation restoration" is an extra text processing after STT. This processing uses additional ML model to guess the right latin punctuation marks (,.?!:) in the text. The model needs to be downloaded via model browser. Right now, it supports only 16 languages (no Arabic support). "Punctuation restoration" is only needed (and only enabled) for DeepSpeech, Vosk and few April models. Whisper and Faster Whisper natively support punctuation therefore this feature is not used for these models. If you are using model from Whisper family, "Punctuation restoration" option (enabled or disables) has not impact on processing.
Regarding "diacritics restoration". It is implemented but not available for text-to-text conversion. When you mark "Restore diacritics before speech synthesis" (enabled by default) and start TTS, input text is processed to restore diacritical marks in the background and output is sent to TTS engine. Right now, you can't see text after diacritics restoration.
Whisper and Faster Whisper natively support punctuation therefore this feature is not used for these models. If you are using model from Whisper family, "Punctuation restoration" option (enabled or disables) has not impact on processing.
My understanding of punctuation is possibly wrong here then.
Arabic can come in 2 formats:
فَرَائِضُ الْوُضُوءِ وَفَرْضُهُ: غَسْلُ الْوَجْهِ، وَغَسْلُ الْيَدَيْنِ مَعَ الْمِرْفَقَيْنِ ، وَمَسْحُ رُبُعِ الرَّأْسِ، وَغَسْلُ الرِّجْلَيْنِ مَعَ الْكَعْبَيْنِ. - diacritics
فرائض الوضوء وفرضه: غسل الوجه، وغسل اليدين مع المرفقين ، ومسح ربع الرأس، وغسل الرجلين مع الكعبين. - no diacritics
Diacritics could be considered punctuation (or vowelized). I assumed that the Whisper models would restore the diacritics as a by-product of punctuation, but the output was like the second example (no diacritics) and hence why I opened this ticket.
Regarding "diacritics restoration". It is implemented but not available for text-to-text conversion. When you mark "Restore diacritics before speech synthesis" (enabled by default) and start TTS, input text is processed to restore diacritical marks in the background and output is sent to TTS engine. Right now, you can't see text after diacritics restoration.
This would be a nice feature to implement. Considering it is already restoring the diacritics for TTS, generating the output for the user could help them see if the model is not hallucinating when reading the text.
Adding to TO-DO list for the next release.
Sorry it took so long.
New version 4.5.0 brings following feature:
The "repair options" are not enabled by default. You have to turn-on "Show Repair text options" in the settings.
When enabled, you can use this:
You need also download additional model:
30695bb7410f173f92751c71579dfd180f22d10f
Thank you so much for implementing this feature and no need to apologize. I am very grateful that you even considered implementing this feature, which would be of limited usage to most other users.
I installed the Whisper medium Arabic model (STT) to test out the punctuation restoration but it isn't clear how to use it.
The button in settings says:
and if enabling it, it says that I need to install the Punctuation model.
But in the Whisper model info, it says:
Is it native in the model or what needs to be done?