search

csdcorp / speech_to_text

A Flutter plugin that exposes device specific text to speech recognition capability.
BSD 3-Clause "New" or "Revised" License
348 stars 217 forks source link

Add '.mixWithOthers' option in audio session category to make speech to text work perfectly when IOS devices are playing audio #488

Closed wamynobe closed 2 months ago

wamynobe commented 2 months ago

relate to #487 #472

wamynobe commented 2 months ago

I'm currently working on Android side

sowens-csd commented 2 months ago

What an excellent PR, thank you for submitting! Are you going to do the Android work under a separate PR or should I wait for that before merging this one?

wamynobe commented 2 months ago

I will update in this pr. It will take me awhile to implement because I am reading the documentation 😁 Then check to make sure it will work great in MacOS also. When I finish, I will tag you here for a review. Thank you!

wamynobe commented 2 months ago

It is quite weird on android. I can not find any API that set the audio in speech to text side. I tried exoplayer with isMixMode option by

exoPlayer.setAudioAttributes(
        new AudioAttributes.Builder().setContentType(C.AUDIO_CONTENT_TYPE_MOVIE).build(),
        !isMixMode);

It worked as I expected. But to the web view case, video or audio is paused all the time when some other features request for audio focus. I have not found any ideas yet.

sowens-csd commented 2 months ago

"It is quite weird on android" I could not agree more!

This is such a useful feature that I think it would be good to get it into a build even if it only supports iOS. If Android is hard we should probably split it into a separate PR and merge this one. Thoughts?

wamynobe commented 2 months ago

@sowens-csd Sure! Let's merge this PR first. I will keep working on the Android platform. Firstly, trying to resolve it from the web view package. I tested it with the package flutter_video which was implemented by exoplayer on Android and It worked perfectly. I think It should be resolved on the web view side.

wamynobe commented 2 months ago

@sowens-csd Can you merge this PR first?