Intermittent Failure of Speech Recognition on iOS (OSStatus error 561017449)

[rodiali1]

When using the expo-speech-recognition library on iOS, speech recognition intermittently fails, mainly when triggering the function for the first time, with the error OSStatus error 561017449. The error seems to occur randomly. The issue is reproducible by either killing the app, reopening it, and then triggering the recognition function, or by waiting for some time and then triggering the recognition.

Things I have tested:

• Manually setting the audio category to 'recording' before starting speech recognition and to 'playback' before and after playing audio.

Environment: Library version: 0.2.13 Expo SDK: 50.0.13 Platform: iOS 17.6.1 Device: iPhone 13

[rodiali1]

I think I’ve figured out the issue!

It looks like the problem with OSStatus error 561017449 was happening because the audio session wasn't being switched properly between recording and playback modes, especially when stopping speech recognition and trying to play audio afterward.

The fix was to delay switching the audio session to playback mode until after speech recognition was fully stopped, in my case I was making an API request, so I triggered the setCategoryIOS after the API call to delay it for some moments . This way, the transition from recording to playback happens smoothly without any conflict.

I think refactoring ExpoSpeechRecognitionModule.stop() into an async function should theoretically fix this.

[jamsch]

Hey @rodiali1, you should be listening to the "end" event so that you can appropriately set the audio category and play back the audio after speech recognition has ended. To be clear, if you want to stop as soon as possible with the least amount of delay before playback, you may want to be using the abort() function as it doesn't take some time to process a final result like the stop() function does.

Unfortunately I don't currently plan to make the stop/abort functions asynchronous as I intend to have the module as realistically close to the Web Speech API spec.

[jamsch]

If you'd like to make the stop/abort functions async you should be able to implement it like this:

const stopAsync = () => {
  return new Promise<void>((resolve, reject) => {
    const listener = addSpeechRecognitionListener("end", handleStop);

    function handleStop() {
      listener.remove();
      resolve();
    }

    // Stop speech recognition
    ExpoSpeechRecognitionModule.stop();
  });
};

[rodiali1]

Hey @rodiali1, you should be listening to the "end" event so that you can appropriately set the audio category and play back the audio after speech recognition has ended. To be clear, if you want to stop as soon as possible with the least amount of delay before playback, you may want to be using the abort() function as it doesn't take some time to process a final result like the stop() function does.

Unfortunately I don't currently plan to make the stop/abort functions asynchronous as I intend to have the module as realistically close to the Web Speech API spec.

Hey @jamsch, thanks for pointing that out! Completely forgot about the end event, but yeah, it worked like a charm moving the set audio category there. Cheers!