search

csdcorp / speech_to_text

A Flutter plugin that exposes device specific text to speech recognition capability.
BSD 3-Clause "New" or "Revised" License
351 stars 218 forks source link

Cant find the workaround for version 2.5 Android 10 stop issue #150

Closed peterweb2005 closed 3 years ago

peterweb2005 commented 3 years ago

as title

after upgraded, found that after 2.4, ie: 2.5, android (10) cant stop as expected, then always flow to the timeout error

peterweb2005 commented 3 years ago

yes, i got, but not help: https://github.com/csdcorp/speech_to_text/pull/131

why the plugin worked in ver 2.4, but not working in ver 2.5?

sowens-csd commented 3 years ago

The difference between 2.4 and 2.5 is that as of 2.5 the code explicitly destroys the Android speech recognizer instead of using the stop method on it. This is because in Android 10 and later the speech recognizer stop doesn't seem to work properly. From your description of the problem I can't tell if that's causing your issue or not. You say it can't stop as expected . What do you expect it to do and what is it doing? It looks like maybe you are getting a timeout? Is this on an emulator or a real device? Can you reproduce the issue on the example app?

peterweb2005 commented 3 years ago

so that is the question

my android 10 only work for stt ver 2.4 or earlier (checked the lock file) my android 10 is sony patch July 2020

may i provide the "debugLogging" message later, whether "Recognizer destroy" appears

sowens-csd commented 3 years ago

I can't tell from your description what problem you're having but I'm wondering if you've hit a different recently fixed issue, #146. Yes, feel free to submit the logs but also include a description of the behaviour and the desired behaviour with it. Also, let me know if the behaviour is the same in the example app.

peterweb2005 commented 3 years ago

thanks, but really checked before open this issue,

my stop() has return empty, as previously worked version behave

peterweb2005 commented 3 years ago

i had "Recognizer destroy" in stop, but no data return

D/SpeechToTextPlugin(23962): Start listening
D/SpeechToTextPlugin(23962): setupRecognizerIntent
D/SpeechToTextPlugin(23962): Notify listening
D/SpeechToTextPlugin(23962): Notify listening done
D/SpeechToTextPlugin(23962): Start listening done
D/SpeechToTextPlugin(23962): In RecognizerIntent apply
D/SpeechToTextPlugin(23962): put model
D/SpeechToTextPlugin(23962): put package
D/SpeechToTextPlugin(23962): put partial
D/SpeechToTextPlugin(23962): put languageTag
D/SpeechToTextPlugin(23962): rmsDB -2.0 / -2.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 8.08
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 8.08
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 8.08
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 8.08
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 8.08
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): Stop listening
D/SpeechToTextPlugin(23962): Notify listening
D/SpeechToTextPlugin(23962): Notify listening done
D/SpeechToTextPlugin(23962): Stop listening done
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(23962): Recognizer destroy
sowens-csd commented 3 years ago

Can you reproduce this problem in the example app?

peterweb2005 commented 3 years ago

of course, just clone to local

Performing hot restart...                                               
Restarted application in 1,002ms.
D/SpeechToTextPlugin(29883): Start initialize
D/SpeechToTextPlugin(29883): Checked permission
D/SpeechToTextPlugin(29883): has permission, completing
D/SpeechToTextPlugin(29883): completeInitialize
D/SpeechToTextPlugin(29883): Testing recognition availability
D/SpeechToTextPlugin(29883): Creating recognizer
D/SpeechToTextPlugin(29883): Setting listener
D/SpeechToTextPlugin(29883): before setup intent
D/SpeechToTextPlugin(29883): setupRecognizerIntent
D/SpeechToTextPlugin(29883): after setup intent
D/SpeechToTextPlugin(29883): sending result
D/SpeechToTextPlugin(29883): leaving complete
D/SpeechToTextPlugin(29883): leaving initializeIfPermitted
D/SpeechToTextPlugin(29883): In RecognizerIntent apply
D/SpeechToTextPlugin(29883): put model
D/SpeechToTextPlugin(29883): put package
D/SpeechToTextPlugin(29883): put partial
I/flutter (29883): ══╡ EXCEPTION CAUGHT BY RENDERING LIBRARY ╞═════════════════════════════════════════════════════════
I/flutter (29883): The following assertion was thrown during layout:
I/flutter (29883): A RenderFlex overflowed by 28 pixels on the right.
I/flutter (29883):
I/flutter (29883): The relevant error-causing widget was:
I/flutter (29883):   Row file:///D:/peter/w_flutter/speech_to_text/speech_to_text/example/lib/main.dart:98:17
I/flutter (29883):
I/flutter (29883): The overflowing RenderFlex has an orientation of Axis.horizontal.
I/flutter (29883): The edge of the RenderFlex that is overflowing has been marked in the rendering with a yellow and
I/flutter (29883): black striped pattern. This is usually caused by the contents being too big for the RenderFlex.
I/flutter (29883): Consider applying a flex factor (e.g. using an Expanded widget) to force the children of the
I/flutter (29883): RenderFlex to fit within the available space instead of being sized to their natural size.
I/flutter (29883): This is considered an error condition because it indicates that there is content that cannot be
I/flutter (29883): seen. If the content is legitimately bigger than the available space, consider clipping it with a
I/flutter (29883): ClipRect widget before putting it in the flex, or using a scrollable container rather than a Flex,
I/flutter (29883): like a ListView.
I/flutter (29883): The specific RenderFlex in question is: RenderFlex#debb1 relayoutBoundary=up3 OVERFLOWING:
I/flutter (29883):   creator: Row ← Column ← Container ← Column ← _BodyBuilder ← MediaQuery ←
I/flutter (29883):     LayoutId-[<_ScaffoldSlot.body>] ← CustomMultiChildLayout ← AnimatedBuilder ← DefaultTextStyle ←
I/flutter (29883):     AnimatedDefaultTextStyle ← _InkFeatures-[GlobalKey#9c723 ink renderer] ← ⋯
I/flutter (29883):   parentData: offset=Offset(0.0, 96.0); flex=null; fit=null (can use size)
I/flutter (29883):   constraints: BoxConstraints(0.0<=w<=320.0, 0.0<=h<=Infinity)
I/flutter (29883):   size: Size(320.0, 48.0)
I/flutter (29883):   direction: horizontal
I/flutter (29883):   mainAxisAlignment: spaceAround
I/flutter (29883):   mainAxisSize: max
I/flutter (29883):   crossAxisAlignment: center
I/flutter (29883):   textDirection: ltr
I/flutter (29883):   verticalDirection: down
I/flutter (29883):   textBaseline: alphabetic
I/flutter (29883): ◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤
I/flutter (29883): ════════════════════════════════════════════════════════════════════════════════════════════════════
D/SpeechToTextPlugin(29883): Start listening
D/SpeechToTextPlugin(29883): setupRecognizerIntent
D/SpeechToTextPlugin(29883): Notify listening
D/SpeechToTextPlugin(29883): Notify listening done
D/SpeechToTextPlugin(29883): Start listening done
D/SpeechToTextPlugin(29883): In RecognizerIntent apply
D/SpeechToTextPlugin(29883): put model
D/SpeechToTextPlugin(29883): put package
D/SpeechToTextPlugin(29883): put partial
D/SpeechToTextPlugin(29883): put languageTag
D/SpeechToTextPlugin(29883): rmsDB -2.0 / -2.0
D/SpeechToTextPlugin(29883): rmsDB -2.0 / 9.5199995
D/SpeechToTextPlugin(29883): rmsDB -2.0 / 9.5199995
I/chatty  (29883): uid=10215(com.csdcorp.speech_to_text_example) identical 2 lines
D/SpeechToTextPlugin(29883): rmsDB -2.0 / 9.5199995
D/SpeechToTextPlugin(29883): rmsDB -2.0 / 10.0
I/chatty  (29883): uid=10215(com.csdcorp.speech_to_text_example) identical 1 line
D/SpeechToTextPlugin(29883): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(29883): rmsDB -2.0 / 10.0
I/chatty  (29883): uid=10215(com.csdcorp.speech_to_text_example) identical 48 lines
D/SpeechToTextPlugin(29883): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(29883): Stop listening
D/SpeechToTextPlugin(29883): Notify listening
D/SpeechToTextPlugin(29883): Notify listening done
D/SpeechToTextPlugin(29883): Stop listening done
D/SpeechToTextPlugin(29883): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(29883): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(29883): Recognizer destroy
peterweb2005 commented 3 years ago

i found that, my android 10 wont need that destroy, 

    private fun stopListening(result: Result) {
        if (sdkVersionTooLow() || isNotInitialized() || isNotListening()) {
            result.success(false)
            return
        }
        debugLog("Stop listening")
        handler.post {
            run {
                speechRecognizer?.stopListening()
            }
        }
        /*if ( !recognizerStops ) {
            destroyRecognizer()
        }*/
        notifyListening(isRecording = false)
        result.success(true)
        debugLog("Stop listening done")
    }

OR just wont stop(), wait speech service to return result

D/SpeechToTextPlugin(31589): Start listening
D/SpeechToTextPlugin(31589): setupRecognizerIntent
D/SpeechToTextPlugin(31589): Notify listening
D/SpeechToTextPlugin(31589): Notify listening done
D/SpeechToTextPlugin(31589): Start listening done
D/SpeechToTextPlugin(31589): In RecognizerIntent apply
D/SpeechToTextPlugin(31589): put model
D/SpeechToTextPlugin(31589): put package
D/SpeechToTextPlugin(31589): put partial
D/SpeechToTextPlugin(31589): put languageTag
D/SpeechToTextPlugin(31589): rmsDB -2.0 / -2.0
D/SpeechToTextPlugin(31589): rmsDB -2.0 / -1.64
D/SpeechToTextPlugin(31589): rmsDB -2.0 / -0.79999995
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 2.56
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 9.5199995
I/chatty  (31589): uid=10215(com.csdcorp.speech_to_text_example) identical 2 lines
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 9.5199995
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
I/chatty  (31589): uid=10215(com.csdcorp.speech_to_text_example) identical 2 lines
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
I/chatty  (31589): uid=10215(com.csdcorp.speech_to_text_example) identical 10 lines
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
I/chatty  (31589): uid=10215(com.csdcorp.speech_to_text_example) identical 43 lines
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(31589): Stop listening
D/SpeechToTextPlugin(31589): Notify listening
D/SpeechToTextPlugin(31589): Notify listening done
D/SpeechToTextPlugin(31589): Stop listening done
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
I/chatty  (31589): uid=10215(com.csdcorp.speech_to_text_example) identical 42 lines
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(31589): Notify listening
D/SpeechToTextPlugin(31589): Notify listening done
D/SpeechToTextPlugin(31589): rmsDB -2.0 / 10.0
D/SpeechToTextPlugin(31589): Calling results callback
I/flutter (31589): result message: 香港 - true
peterweb2005 commented 3 years ago

very wild guess that android 10 has patched the speech problem

so this package should make a flag (eg: destroyOnStop) for developer to choose whether to quickfix the speech problem

sowens-csd commented 3 years ago

Thanks for the logs, I need a bit more detail on what is happening in the code that is not what you expect. A few posts ago you said this "my stop() has return empty, as previously worked version behave". Can you give me a set of steps you use to reproduce the problem like:

  1. Open the example app
  2. Initialize
  3. Call listen
  4. Without speaking first call stop
  5. At this point the application should ... but instead it is ...

This will help me interpret the logs.

peterweb2005 commented 3 years ago

that statement is just response that my issue is NOT the #146 issue

my issue can reproduce just use the "example" app:

  1. Clone the project from Github, edit to my language localeId: 'yue-Hant-HK',
  2. Open the example app through Android Studio in Android 10 running in a Sony phone
  3. Press "Initialize", NO error message
  4. Press "Listen", from log seems start recording
  5. Speak then press "Stop"
  6. Only has a strange beep, sound like cancel beep

After commented the lines which destroy Recognizer, run again through Android Studio, in "flow 5":

  1. Screen display the recognized text: "香港"
sowens-csd commented 3 years ago

Thank you for the detailed description, I understand now.

When you remove the destroy logic do you get a final result when you click stop? What Android used to do after stop was to continue to recognize speech until the timeout was reached.

Try clicking stop and continuing to speak. If the Android bug is fixed then no further words should be shown after you click stop and you should get a final result appearing on screen pretty much immediately. If it is not fixed you will continue to see words being recognized until the time limit is reached or you stop talking and it times out.

I'll check the Android forums to see if there is any discussion of a fix for the issue.

peterweb2005 commented 3 years ago

i wonder if any other one encounter this issue no any other Android 10 mobile seems patched?

sowens-csd commented 3 years ago

I just checked the Android issue and there's no report of a fix. https://issuetracker.google.com/issues/158198432 The issue I was using to track this is #69 which I closed with the work around.

My current theory is that you're seeing the result of my attempted fix for #69 but you didn't answer my question above so I can't tell.

Please tell me exactly what the device does when you comment out the destroy logic and then keep talking after you click stop.

peterweb2005 commented 3 years ago

nothing log after stop then speak

Performing hot restart... Syncing files to device H8324... Restarted application in 1,005ms. D/SpeechToTextPlugin(12928): Start initialize D/SpeechToTextPlugin(12928): Checked permission D/SpeechToTextPlugin(12928): has permission, completing D/SpeechToTextPlugin(12928): completeInitialize D/SpeechToTextPlugin(12928): Testing recognition availability D/SpeechToTextPlugin(12928): sending result D/SpeechToTextPlugin(12928): leaving complete D/SpeechToTextPlugin(12928): leaving initializeIfPermitted I/flutter (12928): _currentLocaleId: en_HK

════════ Exception caught by rendering library ═════════════════════════════════════════════════════ The following assertion was thrown during layout: A RenderFlex overflowed by 28 pixels on the right.

The relevant error-causing widget was: Row file:///D:/peter/w_flutter/speech_to_text/speech_to_text/example/lib/main.dart:99:17 The overflowing RenderFlex has an orientation of Axis.horizontal. The edge of the RenderFlex that is overflowing has been marked in the rendering with a yellow and black striped pattern. This is usually caused by the contents being too big for the RenderFlex.

Consider applying a flex factor (e.g. using an Expanded widget) to force the children of the RenderFlex to fit within the available space instead of being sized to their natural size. This is considered an error condition because it indicates that there is content that cannot be seen. If the content is legitimately bigger than the available space, consider clipping it with a ClipRect widget before putting it in the flex, or using a scrollable container rather than a Flex, like a ListView.

The specific RenderFlex in question is: RenderFlex#8d95c relayoutBoundary=up3 OVERFLOWING ... parentData: offset=Offset(0.0, 96.0); flex=null; fit=null (can use size) ... constraints: BoxConstraints(0.0<=w<=320.0, 0.0<=h<=Infinity) ... size: Size(320.0, 48.0) ... direction: horizontal ... mainAxisAlignment: spaceAround ... mainAxisSize: max ... crossAxisAlignment: center ... textDirection: ltr ... verticalDirection: down ... textBaseline: alphabetic ◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤◢◤ ════════════════════════════════════════════════════════════════════════════════════════════════════ D/SpeechToTextPlugin(12928): Start listening D/SpeechToTextPlugin(12928): setupRecognizerIntent D/SpeechToTextPlugin(12928): Notify listening D/SpeechToTextPlugin(12928): Notify listening done D/SpeechToTextPlugin(12928): Start listening done D/SpeechToTextPlugin(12928): rmsDB -2.0 / -2.0 D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 I/chatty (12928): uid=10215(com.csdcorp.speech_to_text_example) identical 1 line D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 I/chatty (12928): uid=10215(com.csdcorp.speech_to_text_example) identical 32 lines D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 D/SpeechToTextPlugin(12928): Stop listening D/SpeechToTextPlugin(12928): Notify listening D/SpeechToTextPlugin(12928): Notify listening done D/SpeechToTextPlugin(12928): Stop listening done D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 I/chatty (12928): uid=10215(com.csdcorp.speech_to_text_example) identical 21 lines D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 D/SpeechToTextPlugin(12928): Notify listening D/SpeechToTextPlugin(12928): Notify listening done D/SpeechToTextPlugin(12928): rmsDB -2.0 / 10.0 D/SpeechToTextPlugin(12928): Calling results callback I/flutter (12928): result message: 香港 - true

i speak九龍新界after pressed stop

sowens-csd commented 3 years ago

Thanks for the extra details. I will have to either:

  1. Add an option so the developer can choose to use destroy or not
  2. Make the code able to decide if it should use the destroy logic or not.

Based on your report the Android SDK version test it does now is not always right. I'll probably add an option so that at least your code will work without having to modify the plugin source code then think about how to make it smarter.

sowens-csd commented 3 years ago

Version 3.1.0 which is in the repo now has an experimental change that allows the client to specify whether to use destroy or not. Use this to always use stop.

    bool hasSpeech = await speech.initialize(
        onError: errorListener,
        onStatus: statusListener,
        options: [
          SpeechToText.androidAlwaysUseStop,
        ]);

If you have a chance please give it a try and let me know if it resolves your issue.

sowens-csd commented 3 years ago

3.1.0 is now live on pub.dev. If you have any follow-ups on this please open a new issue. Thanks for reporting.