csdcorp / speech_to_text

A Flutter plugin that exposes device specific text to speech recognition capability.
BSD 3-Clause "New" or "Revised" License
351 stars 218 forks source link

locales only has default locale on Android 13 #351

Closed sowens-csd closed 6 months ago

sowens-csd commented 1 year ago

Tested on Pixel 6a also this question on SO about the same issue: https://stackoverflow.com/questions/74295030/query-supported-languages-with-google-speech-services-doesnt-work-on-android-13

newyann666 commented 1 year ago

The fix : https://stackoverflow.com/questions/73516849/how-to-get-devices-default-speechservice-provider-android-api-13

sowens-csd commented 1 year ago

Thanks for following up on this.

Unfortunately that looks like a different issue unless I've misread it. I think the post you linked to is about finding the recognition service rather than the set of supported recognition languages / locales.

newyann666 commented 1 year ago

Thanks for following up on this.

Unfortunately that looks like a different issue unless I've misread it. I think the post you linked to is about finding the recognition service rather than the set of supported recognition languages / locales.

True. On android 13, "speech_to_text" works already like that ? I mean, it search for the default speechservice provider define on the system and use it ?

sowens-csd commented 1 year ago

Yes, it uses the default system defined recognizer which is accessed through the Android createSpeechRecognizer method. There's an option to do a service lookup that uses a similar search mechanism but it has only been useful where the device for some reason doesn't define a default recognizer.

pduong-mipela commented 1 year ago

I got the same issue. On Android it just returns only 1 default language while my app needs to support another language. On iOS, it returns a list of languages. Any workaround please? Or is there a way to install a language on Android?

sowens-csd commented 1 year ago

Unfortunately it's not the language installation that's the problem. Android 13 broke the API that returns the speech recognizer locales. I'll have a look at it again to see if there's a fix available now. The speech recognition will continue to work normally if you know the locale you want to use. As a temporary work around you could define a list of likely locales and show that in the UI with some help for users to install their desired language. Not a great solution I know.

mbcode64 commented 1 year ago

I'm also using the flutter_tts library, and it is able to get a list of installed languages in Android 13. I think it is using android.speech.tts.TexttoSpeech to get the list. Anyway, I just chose the locale that I needed from the returned list and was able to set speech.listen(localeID: locale)

sowens-csd commented 1 year ago

Thanks for the info, that's really helpful. I'll have a look at what they are doing.

sowens-csd commented 1 year ago

So I had a look at TextToSpeech and the Android code uses this method https://developer.android.com/reference/android/speech/tts/TextToSpeech#getAvailableLanguages(). Next question is whether that list matches the list for speech to text. So far I haven't found any confirmation. It seems reasonable to me that it would be the same language pack that provides support for both speech directions.

sowens-csd commented 7 months ago

https://stackoverflow.com/questions/76899610/get-list-of-supported-languages-for-google-speechrecognizer-googlettsrecognitio

sowens-csd commented 7 months ago

New in 33 is https://developer.android.com/reference/android/speech/SpeechRecognizer#checkRecognitionSupport(android.content.Intent,%20java.util.concurrent.Executor,%20android.speech.RecognitionSupportCallback)

which returns this https://developer.android.com/reference/android/speech/RecognitionSupport

which apparently has the list of supported languages. Testing it now.

sowens-csd commented 7 months ago

So in case anyone is watching this thread I have a solution finally. In Android 33+ there is a new API to check recognition support. It returns two different arrays of languages, and only for devices that support on device recognition. The two lists are the list of currently installed languages for the device, and the list of supported languages for the device. So that's a problem. I could return either list from the locales method but they both have their problems. The supported list is probably a better match for the available languages if you use the regular, not on device, recognizer. however the installed list is more accurate if you are using on device recognition. I have chosen to return the supported list after testing and confirming that languages on the supported list can be used with the online recognizer even if not installed. You may need to instruct users to install new language packs on their device if a selected language is not working.

This change is in the repo now and will be released with 6.4.0

Wolfscowl commented 1 week ago

There is an issue with recognitionSupport.onlineLanguages The list is always empty.

    fun test() {
        if (Build.VERSION.SDK_INT > 32) {
            val speechRecognizer = SpeechRecognizer.createSpeechRecognizer(app)
            val intent = Intent(RecognizerIntent.ACTION_GET_LANGUAGE_DETAILS)
            speechRecognizer.checkRecognitionSupport(intent, Executors.newSingleThreadExecutor(),
                object : RecognitionSupportCallback {
                    override fun onSupportResult(recognitionSupport: RecognitionSupport) {
                        Log.d(currentFileName(), "API-33 Support onlineLanguages " + recognitionSupport.onlineLanguages)
                        Log.d(currentFileName(), "API-33 Support onDeviceLang " + recognitionSupport.supportedOnDeviceLanguages)
                    }
                    override fun onError(error: Int) {
                        Log.d(currentFileName(), "API-33 ErrorCode $error")
                    }
                })
        }
    }

The output looks strange. onSupportResult is called twice and onError once

API-33 ErrorCode 14
API-33 Support onlineLanguages []
API-33 Support onDeviceLang [de-AT, de-BE, de-CH, de-DE, en-AU, en-CA, ...
API-33 Support onlineLanguages []
API-33 Support onDeviceLang [de-AT, de-BE, de-CH, de-DE, en-AU, en-CA, ...

any solution?