Speech.stop() does not immediately stop recording speech in Galaxy S9 and S9+ Phones

[kuranfmital]

When calling the speech.stop() function on a Galaxy S9 or S9+ phone running Android 10, there is a ~5 second delay between when onStatus is notified that the microphone has turned off, and when the the microphone stops recording speech.

For example:

class SpeechManager {

  stt.SpeechToText speech = stt.SpeechToText();
  bool _canListen;

  Future<void> initialize() async {
     this._canListen = await speech.initialize(onStatus: this._statusListener);
   }

   void _statusListener(String status) {
      if (status == "listening") {
         print('Mic is now listening');
      }
      if (status == "notListening") {
         print('Mic is no longer listening');
      }
   }

   Future<void> onPressOn() async {
      await speech.listen(onResult: callback(), cancelOnError: true);
   }

   Future<void> onPressOff() async {
      await speech.stop();
   }
}

If I press a button which triggers the onPressOff() function, the console will immediately log "Mic is no longer listening," but the microphone will continue to pick up speech for several seconds before the speech_to_text_stop.m4r audio plays and the microphone stops recording.

This same behavior occurs if we trigger speech.cancel() in place of speech.stop() .

This issue does not appear when running on older Samsung Galaxy phones, or on a Pixel 3A running Android 10.

[sowens-csd]

That's really odd. Can you try initializing with the debugLogging: true flag set and check the Android logs to see how the timing on the calls to the speech recognition intent on Android line up with the stop behaviour? I'm curious if the plugin is being slow to invoke the Android methods, or if Android is being slow stopping listening.

[sowens-csd]

Any more information on this?

[kuranfmital]

Here is an annotated breakdown of the logs:

04-22 14:12:29.139 3663 3859 D InputReader: Input event(4): value=1 when=14495826692000 04-22 14:12:29.139 3663 3859 D InputReader: Input event(4): value=1 when=14495826692000 04-22 14:12:29.139 3663 3859 I InputReader: Touch event's action is 0x0 (deviceType=0) [pCnt=1, s=0.1470 ] when=14495826692000

04-22 14:12:29.172 3663 3992 D lights : button : 1 -

04-22 14:12:29.193 30329 30528 I Icing : Internal init done: storage state 0

04-22 14:12:29.207 29116 29152 I flutter : Stopping listen (This is right after the mic button is tapped to stop listening)

04-22 14:12:29.215 30329 31067 I Icing : IndexChimeraService.getServiceInterface callingPackage=com.google.android.gms componentName=AppsCorpus serviceId=32

04-22 14:12:29.245 29116 29152 I flutter : Stopped (This is when the status changed to "notListening")

04-22 14:12:29.284 30329 30528 I Icing : Post-init done

04-22 14:12:29.288 3663 3712 D CompatibilityInfo: mCompatibilityFlags - 0

04-22 14:12:29.297 5463 30927 E AudioRecord-JNI: Error -4 during AudioRecord native read

04-22 14:12:29.302 5463 30929 W b : #read failed: read EOF from AudioAccessor. 04-22 14:12:29.302 5463 30936 W b : #read failed: read EOF from AudioAccessor. 04-22 14:12:29.302 5463 31003 W b : #read failed: read EOF from AudioAccessor.

04-22 14:12:29.321 3175 30932 V audio_hw_primary: select_devices() output_route "media-speaker" 04-22 14:12:29.321 3175 30932 I audio_route: > audio_route_apply_path : "gain-media-speaker" 04-22 14:12:29.321 3175 30932 V audio_hw_primary: select_devices() output_gain "gain-media-speaker"

04-22 14:12:29.321 3175 30932 I audio_route: > audio_route_update_mixer : +

04-22 14:12:29.328 5463 31009 I AudioStreamSource: Audio size: 7426 bytes

04-22 14:12:29.354 30329 30528 I GmsCoreXrpcWrapper: Returning a channel provider with trafficStatsTag=12803

04-22 14:12:29.367 5463 5588 I AudioRequestLSAdapter: StopListeningStatus result: 1 (I believe this is when it has actually stopped listening)

04-22 14:12:29.384 5463 5917 I AudioRequestLSAdapter: StopListeningStatus result: 1

[kuranfmital]

After building the same application for IOS, we are seeing the same behavior on IOS where turning off the microphone has a delay before actually turning off

[sowens-csd]

Your original post said ~5 seconds but that log shows about 122 ms delay between the two states. It should probably only confirm after it has stopped listening but I would like to understand if you're seeing a 5 second delay because if so this differential wouldn't explain it.

[sowens-csd]

Any update on this?

[chernandez7]

That's correct, log said 122ms but we still saw ~5 second delay on the actual speech detection. Hard to give an update since we don't have access to a Galaxy S9 or iPhone to make a reproducible demo.

[sowens-csd]

Unfortunately I can't reproduce this and I don't have enough information to track it down. I'll close this with a cannot reproduce but please reopen it or create a new issue if you find out anything else.

[aaka3207]

I'm also having this issue, tested on multiple devices. I've noticed it happening mainly on devices running Android 9 and 10, wondering if it has anything to do with this issue https://issuetracker.google.com/issues/155690736#comment2

[sowens-csd]

It sounds like it could well be related. There's also this issue which is active very recently. https://issuetracker.google.com/issues/158198432

I'll track that issue and see if there is any resolution. I'll leave this issue open to track it. Thanks for reporting!

[sathishmscict]

Nokia 6.1 plus OS: Android 10

I have been faced same issue on my device. Speech.stop() action performed than after also text recognizing.

SpeechToTextProvider.soundLevelListener: sound level 4.600000381469727: -0.31999993324279785 - 10.0 I/flutter (23843): 2020-06-26T11:50:03.591209 D SpeechToTextProvider.soundLevelListener: sound level 2.320000171661377: -0.31999993324279785 - 10.0 I/flutter (23843): 0:00:11.484360 D SpeechToTextProvider.soundLevelListener: sound level 2.320000171661377: -0.31999993324279785 - 10.0 E/AccessibilityBridge(23843): VirtualView node must not be the root node. I/flutter (23843): 2020-06-26T11:50:03.654827 D SpeechToTextProvider.statusListener: Received listener status: notListening, listening: false I/flutter (23843): 0:00:11.547982 D SpeechToTextProvider.statusListener: Received listener status: notListening, listening: false E/AccessibilityBridge(23843): VirtualView node must not be the root node. I/flutter (23843): 2020-06-26T11:50:03.883126 D SpeechToTextProvider.resultListener: result : SpeechRecognitionResult words: [SpeechRecognitionWords words: 123456789, confidence: -1.0], final: false I/flutter (23843): 0:00:11.776261 D SpeechToTextProvider.resultListener: result : SpeechRecognitionResult words: [SpeechRecognitionWords words: 123456789, confidence: -1.0], final: false E/AccessibilityBridge(23843): VirtualView node must not be the root node. I/flutter (23843): 2020-06-26T11:50:04.154310 D SpeechToTextProvider.resultListener: result : SpeechRecognitionResult words: [SpeechRecognitionWords words: , confidence: -1.0], final: false

Status seems: notListenting, but actually its listening in my device.

[sowens-csd]

Looks like as of July 7th the issue has been assigned to engineering team at Google, so that's a hopeful sign.

[sowens-csd]

Just checked in, no response to a request for an update on July 21st.

[sowens-csd]

The Android bug still has no changes so I've attempted a mitigation in speech_to_text.

If you have a chance could you try the android_destroy branch? This mitigation destroys the speech recognizer intent instead of using the stopListening/cancel methods that don't appear to work. It would mean that the recognizer would stop immediately. The difference between this and using the Android stopListening method is that stopListening finishes processing outstanding speech and reports final results before terminating. Destroy terminates immediately which means that it won't finish processing. Due to a change I made for a bug in iOS that meant it sometimes didn't return final results the speech_to_text library will generate a final if none appear after stop is called. That final will be the same as the last interim result returned from the OS. So, not quite as good as having a working stop but it would remove the delay. This version does both call stopListening, delays for 50 ms and then destroys the recognizer. On Android 9 that gives time for a final result, not sure what it will do on 10.

This version of the fix always destroys, a more complete implementation would probably either check OS version or accept a flag on the initialize method so that it would not destroy the recognizer on OS versions that work. If I get positive feedback on this version I'll complete that work and release it.

[sowens-csd]

New version just committed on the same branch. There was a mistake in the previous version that meant it wasn't actually destroying the recognizer. Let me know if anyone has time to try it again.

[sowens-csd]

There's a final version of this fix on main now if anyone wants to give it a try.

[aaka3207]

@sowens-csd I will be trying this over the next few days, will keep you posted.

[aaka3207]

@sowens-csd Sorry for the late reply, this seems to be working though.

[sowens-csd]

I'll keep monitoring that Google issue but close this issue as resolved with my work around.