CommunityToolkit / Maui

The .NET MAUI Community Toolkit is a community-created library that contains .NET MAUI Extensions, Advanced UI/UX Controls, and Behaviors to help make your life as a .NET MAUI developer easier
https://learn.microsoft.com/dotnet/communitytoolkit/maui
MIT License
2.22k stars 385 forks source link

[BUG] SpeechToText on iOS 17 cuts off or fails to recognize many words #1966

Open AlessandroLanzoni1991 opened 3 months ago

AlessandroLanzoni1991 commented 3 months ago

Is there an existing issue for this?

Did you read the "Reporting a bug" section on Contributing file?

Current Behavior

I am currently experiencing an issue with SpeechToText on iOS 17 where during voice recognition, certain words are either cut off or not recognized correctly. For example: "Alessandro" become Al or Alex. "Planification" become "plan". "Configuration" become "config"

Expected Behavior

All spoken words should be transcribed accurately without being cut off or distorted during speech recognition.

Steps To Reproduce

  1. Open any application that uses SpeechToText functionality on an iOS 17 device.
  2. Speak various sentences and words.
  3. Notice how many words are truncated or inaccurately transcribed during the speech recognition process, for example Names

Link to public reproduction project repository

https://github.com/jfversluis/MauiSpeechToTextSample

Environment

- .NET MAUI CommunityToolkit: 9.0.1
- OS: 17.5.1    / Iphone 12
- .NET MAUI: 8.0.60

Anything else?

This works well on my iPhone 8 whit iOS 16.7.6

vhugogarcia commented 3 months ago

Thanks @AlessandroLanzoni1991 for your feedback. I was able to verify the issue. However, I noticed that it is an expected result as it is right now configured.

When creating a new SFSpeechAudioBufferRecognitionRequest(); in the code below, Apple by default sets the ShouldReportPartialResults to True. I tested it by running the sample project of the MCT and setting that property to false and it worked OK. The only downtime of that is that it has a delay which prevents to show the text in real time.

By saying that, I think it will be ideal to add a property where we can set that to True or False based on the user's requirements.

https://github.com/CommunityToolkit/Maui/blob/819ab10a3a53c76e514410e8ee0af2c5c7a6fe9c/src/CommunityToolkit.Maui.Core/Essentials/SpeechToText/SpeechToTextImplementation.ios.cs#L24

Thanks

Spoxeman commented 1 month ago

I am experiencing the same issues, Is there any resolution yet?

@vhugogarcia ..I cloned the ios implementation and made the changes to set ShouldReportPartialResults to false and it works, no more truncation. How do we get this into release?

VladislavAntonyuk commented 3 days ago

Working on it

aaldrichaes2 commented 2 days ago

Is there a workaround for this currently?