search

google / oboe

Oboe is a C++ library that makes it easy to build high-performance audio apps on Android.
Apache License 2.0
3.64k stars 554 forks source link

LiveEffect sample: is there any way to get input stream from default/incoming VOIP call audio source instead of only microphone #2058

Closed huynhtanloc2612 closed 2 weeks ago

huynhtanloc2612 commented 3 weeks ago

Hi Oboe developers, Thank you for making great library. I have been trying LiveEffect sample and very impressive about very low latency from microphone to speaker. I am currently have a question: instead of taking the inputstream from only microphone, is there any way to get input stream from default audio source (like MediaRecorder.AudioSource.Default) which is including all audio sound from a phone or get input stream from MediaRecorder.AudioSource.VOICE_CALL/MediaRecorder.AudioSource.VOICE_DOWNLINK.... audio sources? I have been searching about this on guidelines and internet but have not found the way yet. Please tell me if you have any idea. Thanks.

philburk commented 3 weeks ago

In general, Android does not allow capture of Voice Communication streams for privacy reasons.

But this API may help you: https://developer.android.com/media/platform/av-capture

And also see this AAudio call: https://developer.android.com/ndk/reference/group/audio#aaudiostreambuilder_setallowedcapturepolicy

huynhtanloc2612 commented 3 weeks ago

Thanks @philburk for your reply and suggestions.

I tried MediaProjection and MediaRecorder before. My purpose is to show phone screen on TV via chromecast and output voice call audio via speaker of TV in Meet/Duo session. Above API could help capture both screen and voice audio streams. Then I stream the captured data to chromecast via http server. However latency was high (around 5-10 seconds) which was not working for a call.

After trying LiveEffect sample which shows impressive low latency from capturing microphone audio stream to playing the stream on speaker, I think if I could capture the voice audio stream (with root permission) and playback stream as a media stream in the same way as LiveEffect sample. This means I could convert voice audio stream to media stream with low latency as what I am looking for in link. Then I could use "mirror screen" feature of Google Home app which can mirror screen and output media audio on TV via chromecast.

Studying the guideline of Oboe API, I am seeing that it supports capturing streams from input devices something similar to Mediarecorder. But I am not sure if it is possible to capture voice audio stream even with root permission or some modification in code.

Sorry if my purpose and question are out of the scope of your support. Really appreciated if you have any ideas/suggestions. Thanks!

philburk commented 2 weeks ago

Oboe/AAudio can capture from the microphone with low latency. But that stream is muted if a VoiceCommunication stream or phone call is active.

huynhtanloc2612 commented 2 weeks ago

Thanks @philburk It currently seems that phone cannot route call voice audio stream to TV via chromecast devices.