search

ant-media / Ant-Media-Server

Ant Media Server is a live streaming engine software that provides adaptive, ultra low latency streaming by using WebRTC technology with ~0.5 seconds latency. Ant Media Server is auto-scalable and it can run on-premise or on-cloud.
https://antmedia.io
Other
4.29k stars 632 forks source link

Use system audio in Screen Share with Android WebRTC SDK #2940

Open SelimEmre opened 3 years ago

SelimEmre commented 3 years ago

I want to use system audio in a screen share. Android WebRTC SDK uses microphone audio by default.

Ticket ID: 38666

thinkgopal commented 3 years ago

Hi Selim,

Have you found anything to achieve this?

mekya commented 3 years ago

Thank you for follow up @thinkgopal.

There is no update about this issue. Do you have any chance to help us? So that we can create a quick solution for you. Do you know if Android allows an application to capture the System Audio and having some references/links?

thinkgopal commented 3 years ago

@mekya Based on stackoverflow threads, some are telling it's possible to record internal audio of Android device. Please check below links for reference. https://stackoverflow.com/questions/17676142/record-android-audio-output https://www.androidpolice.com/2020/06/07/record-audio-android-apps/

mekya commented 3 years ago

Yeah it seems that there is a support with Android 10.

Thank you. We can put this request to our roadmap. Let me plan when we can make it.

prabhat-oodles commented 1 year ago

Yeah it seems that there is a support with Android 10.

Thank you. We can put this request to our roadmap. Let me plan when we can make it.

Hi @mekya, what is the progress on it. I am using latest ant media Android sample app and It isn't using system audio.

mekya commented 1 year ago

Hi @prabhat-oodles ,

Thank you for follow up. AFAIK it is implemented and should be working. On the other hand, this issue is still open. It seems that there is a confusion somewhere.

I'm putting it on the week's schedule to check it if we have some availability.

prabhat-oodles commented 1 year ago

Hi @prabhat-oodles ,

Thank you for follow up. AFAIK it is implemented and should be working. On the other hand, this issue is still open. It seems that there is a confusion somewhere.

I'm putting it on the week's schedule to check it if we have some availability.

Hi @mekya, I have downloaded the latest sample and using Samsung M31 (Android 13) device to test it. But, I am unable to stream my device audio. Only the sound from my mic is getting streamed when I share screen. Kindly check the current implementation.

PS: I forgot to mention that I am facing this issue when I try to switch from camera to Screen Capture while live streaming. If I start the stream using the share screen feature then the device audio is streaming fine.

mekya commented 1 year ago

Thank you @prabhat-oodles for the update

prabhat-oodles commented 1 year ago

Thank you @prabhat-oodles for the update

Hi @mekya I have edited my above comment for more information.

PS: I forgot to mention that I am facing this issue when I try to switch from camera to Screen Capture while live streaming. If I start the stream using the share screen feature then the device audio is streaming fine.

mekya commented 1 year ago

Hi @prabhat-oodles,

Thank you for the update. My colleague, @lastpeony, is going to work on this issue in this week.

@lastpeony , please pay attention to the @prabhat-oodles comment above.

Take care Oguz

lastpeony commented 1 year ago

Mentioned PR feature enables switching of audio sources during streaming. When a user transitions from using the camera to screen sharing, the audio recording automatically switches from the microphone to system audio. Likewise, when transitioning from screen sharing back to the camera, the audio seamlessly switches back from system audio to the microphone.

prabhat-oodles commented 1 year ago

Hi @lastpeony. What is the latest update on the above issue? It is still showing in progress.

lastpeony commented 1 year ago

Hi @prabhat-oodles Good news! The reported issue has been resolved, and the desired feature has been successfully implemented. You can begin utilizing it by checking out the branch associated with this pull request: GitHub Link. The changes from this branch will soon be merged into the master branch.

prabhat-oodles commented 1 year ago

Hi @lastpeony, downloaded code from your branch and added it in my project. Received below error while starting the stream.

java.lang.AssertionError: Expected condition to be true at org.webrtc.audio.WebRtcAudioEffects.assertTrue(WebRtcAudioEffects.java:194) at org.webrtc.audio.WebRtcAudioEffects.enable(WebRtcAudioEffects.java:109) at org.webrtc.audio.WebRtcAudioRecord.initRecording(WebRtcAudioRecord.java:333) --------- beginning of crash #

Fatal error in: gen/jni_headers/sdk/android/generated_metrics_jni/../../../../../../../sdk/android/src/jni/jni_generator_helper.h, line 94

last system error: 0

Check failed: !env->ExceptionCheck()

Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 12276 (worker_thread -), pid 9871

Stream crashed after this error

lastpeony commented 1 year ago

Hi @prabhat-oodles I just checked out the branch again and launched ScreenCaptureActivity I was able to succesfully use the new feature. Did you check the changes on ScreenCaptureActivity and modified your own applications activity acordingly?

prabhat-oodles commented 1 year ago

Hi @prabhat-oodles I just checked out the branch again and launched ScreenCaptureActivity I was able to succesfully use the new feature. Did you check the changes on ScreenCaptureActivity and modified your own applications activity acordingly?

Hi @lastpeony , I have made changes to my own app and also ran the sample at my end. In both scenarios, the streaming is also crashing with the same above error. Do I need to do anything else at my end other than the changes in the ScreenCaptureActivity?

prabhat-oodles commented 1 year ago

Hi @lastpeony ,

effects.enable(audioRecord.getAudioSessionId());

The above line is getting called twice in WebRtcAudioRecord.java. Once inside createAudioRecord() and then again inside initRecording() function. On second call, the assertion inside effects.enable() fails with above error.

This could be the issue, could you please check it at your end?

lastpeony commented 1 year ago

@prabhat-oodles I just reproduced and will update you soon.

lastpeony commented 1 year ago

@prabhat-oodles Could please check it again? I sent a fix and tested it on Android 13 emulator. When i switch to screen share it shares system audio and when i switch back to cameras it shares microphone audio.

prabhat-oodles commented 1 year ago

@prabhat-oodles Could please check it again? I sent a fix and tested it on Android 13 emulator. When i switch to screen share it shares system audio and when i switch back to cameras it shares microphone audio.

Thanks for quick response @lastpeony. I will check it and let you know. In the meantime, you can check another issue if possible. The streaming also crashes if we switch camera and share screen before starting the stream and then start. It gives the error - Could not register audio record policy