iOS start listening sound not playing on real device (iPhone 7 iOS 13.4.1)

csdcorp / speech_to_text

A Flutter plugin that exposes device specific text to speech recognition capability.

BSD 3-Clause "New" or "Revised" License

351 stars 218 forks source link

iOS start listening sound not playing on real device (iPhone 7 iOS 13.4.1) #70

Closed awilliams684 closed 4 years ago

awilliams684 commented 4 years ago

Stop and cancel sounds play fine, however the start listening sound is not playing on a real device. Using the example project, confirmed all sounds DO play in simulator.

sowens-csd commented 4 years ago

Looks like I might be changing the audio state too quickly after playing the start sound so it's getting cutoff. Might have to wait for the end of the playback before starting the listen session. I'll have a look. Thanks for reporting.

awilliams684 commented 4 years ago

That sounds like a possibility, I was having similar problems using audio players package in flutter to try and play a start listening tone.

If the listening function was called before the player was finished it would get cutoff as you described.

sowens-csd commented 4 years ago

Okay, there's a version in the repo now that I'd like you to try. It now works on my device, didn't before the change.

I think the problem was that I was resetting the audio session mode directly after playing the sound and that was interrupting it. Assuming this works for you it's a less disruptive change than waiting until the audio plays to start listening.

awilliams684 commented 4 years ago

Update...

Tested on the following devices: iPhone8 iOS 13.3.1 iPhone7 iOS 13.4.1 iPad4 iOS 10.3.4

Sounds work on ALL devices, however on both the iPhone7 and iPhone8 the volume of the start listening sounds is extremely low and barely audible, the stop and cancel sounds play at normal volume. The iPad seems to work as expected.

Here is a video of the behavior (sorry for the background noise)

https://ta-laboratories.com/videos/Video.mov

awilliams684 commented 4 years ago

Just did a little digging and confirmed the start listening audio is playing from the earpiece and NOT the speaker on the iPhones. Looks like the default audio route needs to be over-ridden to force playback from external speaker on the phones. That would explain why the ipad is working correctly due to it lacking an earpiece.

Found this post describing the issue https://stackoverflow.com/questions/29625633/avaudioplayer-playing-sound-with-very-low-volume-in-iphone-6-and-6

awilliams684 commented 4 years ago

Changing line 249 will override the audio route, just tested and this works on the phone, volume is still lower than the stop and cancel sounds but it is coming from the external speaker now.

Line 249 original: try self.audioSession.setCategory(AVAudioSession.Category.playAndRecord)

Line 249 modified: try self.audioSession.setCategory(AVAudioSession.Category.playAndRecord, options: .defaultToSpeaker)

awilliams684 commented 4 years ago

Ok, I fixed the volume issue but changing the AVAudioSession mode from measurement to default on line 251. Confirmed this is working on my physical device!

Line 251 before: try self.audioSession.setMode(AVAudioSession.Mode.measurement)

Line 251 after: try self.audioSession.setMode(AVAudioSession.Mode.default)

sowens-csd commented 4 years ago

Thanks for checking it so quickly, great to hear that it's working.

For the issue with the sound coming from the headphones, I would have thought that would be the desired behaviour if headphones are plugged in? I don't think that the plugin should force sound to come from the speaker if there are headphones plugged in. Maybe I'm not understanding the issue properly.
The volume issue is interesting. I initially changed to measurement rather than default as part of an approach to being able to implement the onSoundLevelChange callback for iOS that is already working on Android. I'm not sure it's required and if it affects volume perhaps there's another approach. Thanks for the research!

awilliams684 commented 4 years ago

I would agree that it would be preferable for sounds to come from headphones or Bluetooth audio device if in use. I was experiencing the start listening tone coming from the phones built in earpiece speaker that is used for normal phone calls.

The solution of overriding the route is admittedly probably not the best way to correct this, but for now it will have to do in my application. I hope the information can help you figure out a better solution.

sowens-csd commented 4 years ago

Thanks for the video, that was really helpful. I want to make sure I understand what's happening on the phone. When you said 'earpiece' I thought you meant headphones. However, after watching your video I think you mean the top speaker that you'd use with the phone against your ear? (Is there an official name for that?) So the goal would be to make the sound come from the speakers on the phone used for music or when in speakerphone mode, not the top speaker used only for phone mode. This should be true only when there are no headphones plugged into the phone.

Can you confirm that's what you were looking for? Sorry I misunderstood your first description, I should have watched the video earlier.

sowens-csd commented 4 years ago

And after reading the description of what .defaultToSpeaker does it looks like that's exactly right. I'm going to make this change, thanks for the help!

sowens-csd commented 4 years ago

I just pushed a new version to the repo that I think incorporates all of these changes safely. Stopping the audio session caused problems but I think I've tamed it. If you have a chance and could give the repo version a try it would be helpful, thanks!

sowens-csd commented 4 years ago

This is resolved with the 2.2.0 release to pub.dev.