search

vilicvane / cordova-plugin-tts

Cordova Text-to-Speech Plugin (Maintainer WANTED!)
176 stars 137 forks source link

Highlight spoken words #51

Open Sam747 opened 6 years ago

Sam747 commented 6 years ago

I successfully integrated tts plugin. I would like to know if somehow I can highlight currently spoken words on my front-end. In other words if it could simultaneously speak and output as in text form what is currently being spoken.

Thanks

billism1 commented 6 years ago

Currently, that is not implemented in this Cordova plugin. The functionality is there in the underlying target APIs for Android and iOS. I haven't looked into Windows Phone.

For this to work in TTS, it would have to expose or encapsulate the proper callbacks/delegates.

In iOS, it looks like we'd probably be making use of AVSpeechSynthesizerDelegate and speechSynthesizer(_:willSpeakRangeOfSpeechString:utterance:)

https://developer.apple.com/documentation/avfoundation/avspeechsynthesizerdelegate

https://developer.apple.com/documentation/avfoundation/avspeechsynthesizerdelegate/1619681-speechsynthesizer

In Android, it looks like we'd be dealing with SynthesisCallback.rangeStart and UtteranceProgressListener.onRangeStart

https://developer.android.com/reference/android/speech/tts/SynthesisCallback.html#rangeStart(int, int, int)

https://developer.android.com/reference/android/speech/tts/UtteranceProgressListener.html#onRangeStart(java.lang.String, int, int, int)

It would be nice to have this functionality on a project I may be working on. If so, when the time comes, if it hasn't been added by someone, yet, I may give it a try.