csdcorp / speech_to_text

A Flutter plugin that exposes device specific text to speech recognition capability.
BSD 3-Clause "New" or "Revised" License
351 stars 218 forks source link

Question: Capitalize the first word in every sentence? #303

Closed asmith26 closed 2 years ago

asmith26 commented 2 years ago

Hi there,

I was wondering if it is possible to automatically capitalize the first word in every sentence, e.g. when I say:

"hello world FULLSTOP how are you QUESTION MARK"

  • I get: hello world. how are you?
  • My preferred output: Hello world. How are you?

Thanks for any help, and for this fantastic lib! :)

sowens-csd commented 2 years ago

Have you noticed this on both iOS and Android? I was just having a look at the iOS code and it uses the formattedString output of the speech recognizer. I just checked the example app and it appears to properly capitalize the beginning of each sentence as long as I add punctuation in the dictation.

asmith26 commented 2 years ago

Thanks for the info @sowens-csd. I've only tried this on Android - which example are you referring too? I'm currently using the following code https://pub.dev/packages/speech_to_text#complete-flutter-example

sowens-csd commented 2 years ago

Interesting, I'll try this on Android. Thanks!

Embarrassingly I hadn't even noticed capitalization as it didn't matter for my particular use case.

asmith26 commented 2 years ago

Embarrassingly I hadn't even noticed capitalization as it didn't matter for my particular use case.

No problem at all, thanks for your help and for this amazing lib!

sowens-csd commented 2 years ago

So I tested this on Android and sure enough it doesn't return any capitals. I've checked the docs and I've seen no mention of it. There is, as far as I can tell only one way to get the results and it doesn't return any capitalization, unlike on iOS. For now at least it looks like this is a difference between the two versions that I can't easily bridge. I don't see anything in any of the support forums about it so it appears to just not be supported on Android. If anyone has any ideas I'd be happy to hear them.

I did try dictating from the Android keyboard in Google docs and it capitalized properly as it does in Gmail. Right now I'm thinking that dictation is adding extra processing after the voice recognition but I'm not certain.

asmith26 commented 2 years ago

Here is my initial idea (this currently only capitalizes the very first letter, not the first letter of each sentence - I'm still learning Dart :)

extension stringExtension on String {
  String get capitalize => "${this[0].toUpperCase()}${this.substring(1)}";
}

...
result.recognizedWords.capitalize
...
sowens-csd commented 2 years ago

So last night, I had a thought, that really should have occurred to me before. There are various dictation modes you can use in the listen method. In confirmation mode, which is what the example ships with, Android doesn't capitalize. However, if you put it in dictation mode it does, at least mostly. The first word of the first sentence isn't capitalized unless it's a proper noun or other obviously capitalized word. However it does capitalize each subsequent sentence, proper nouns, etc. So I think that solves it. To modify the example app change the listen statement to this:

speech.listen(
        onResult: resultListener,
        listenFor: Duration( seconds: 30),
        pauseFor: Duration( seconds: 3),
        partialResults: true,
        localeId: _currentLocaleId,
        onSoundLevelChange: soundLevelListener,
        cancelOnError: true,
        listenMode: ListenMode.dictation);
asmith26 commented 2 years ago

Thanks very much for your help @sowens-csd. Unfortunately when I try using listenMode: ListenMode.dictation in this example I still get hello world. how are you?

What's even more bizarre, when I don't pronouce "world" that well, and the speech recognition engine predicts "wild", I get hello wild. How are you?

asmith26 commented 2 years ago

I think I've found a solution:

extension stringExtension on String {
  String get capitalize => "${this[0].toUpperCase()}${this.substring(1)}";
  String get capitalizeFullStop => "${this.split('. ').map((sentence) => sentence.capitalize).join('. ')}";
  String get capitalizeQuestionMark => "${this.split('? ').map((sentence) => sentence.capitalize).join('? ')}";
  String get capitalizeExclamationMark => "${this.split('! ').map((sentence) => sentence.capitalize).join('! ')}";
  String get capitalizeSentences => "${this.capitalizeFullStop.capitalizeQuestionMark.capitalizeExclamationMark}";
}

void main() {
  print("hello world. how are you? it's a great day! more words. some more words! even more words!".capitalizeSentences);
}

More beautiful solutions very welcome :)

sowens-csd commented 2 years ago

It's odd that the Android capitalization isn't working well for you, I suppose it may be device or OS version specific for some of it. In my tests other than the first word spoken it seems to be capitalizing everything properly.

In any event I don't think the plugin should help with capitalization, if the underlying speech recognition isn't working properly then the app would have to decide what capitalization it needs and implement that. I think we can probably close this issue as the plugin is properly passing through whatever capitalization the platform provides.

asmith26 commented 2 years ago

That makes sense, many thanks again for all your help!