search

pedroSG94 / RootEncoder

RootEncoder for Android (rtmp-rtsp-stream-client-java) is a stream encoder to push video/audio to media servers using protocols RTMP, RTSP, SRT and UDP with all code written in Java/Kotlin
Apache License 2.0
2.57k stars 778 forks source link

Audio mix source #1597

Open noho501 opened 1 month ago

noho501 commented 1 month ago

Hi @pedroSG94 ,

Do you have any plans to improve the sound mixing quality when using both the microphone and in-app sound? https://github.com/pedroSG94/RootEncoder/pull/1571

Currently, if both are used, the sound is not very clear. It seems like the microphone is picking up sound from both the speaker and microphone, making the live stream sound noisy and unpleasant.

If there is a way to mix the app sound and microphone audio more transparently, it would be great (I have already enabled echo cancellation and noise cancellation). I tried using the Larix app, and there is a noticeable difference in the sound quality.

Best regards,

pedroSG94 commented 1 month ago

Hello,

What do you means with noisy and unpleasant? Do you have audio pops? echo? Can you reproduce it in all devices?

noho501 commented 1 month ago

Yes, that's correct. I have tested both the RootEncoder and Larix apps, and below are the test links. The RootEncoder does not perform as well as Larix in terms of sound quality.

Larix: https://youtube.com/live/9tu5ujjUPxk
RootEncoder: https://youtube.com/live/ReBu0ijdaVc

pedroSG94 commented 1 month ago

I see, I will check it.

For now, try add this line after create the stream class:

setTimestampMode(TimestampMode.BUFFER, TimestampMode.BUFFER)

This is only available in the master branch so you can use the app example with screen example and add that line here: https://github.com/pedroSG94/RootEncoder/blob/master/app/src/main/java/com/pedro/streamer/screen/ScreenService.kt#L89

pedroSG94 commented 1 month ago

I was doing tests using MixAudioSource and the audio sometimes produce noise but in general the quality is fine in my case. I tested using Pixel 6a and Xioami A3 (Android 14 and Android 10).

Also, after use TimestampMode.BUFFER, the quality is better and then I can't reproduce audio noise so this seem to solve the problem.

Let me know if you was able to fix the audio using that.

noho501 commented 1 month ago

Hi,

We will run a test and provide you with the results. Thank you for your help!

noho501 commented 1 month ago

Hi,

Thank you for your help. It seems to be working better than before.

noho501 commented 2 weeks ago

Hi @pedroSG94 ,

We've encountered an issue with screen broadcasting that occurs after 20 minutes. At that point, an audio delay appears; for example, gunshots sound before they are visible in the video. You can see an example here: Delay.

Additionally, there's noticeable noise when the microphone is on, even though Noise Cancellation is enabled. Here’s an example: Noise.

Do you have any suggestions on how to address these issues?

pedroSG94 commented 2 weeks ago

Hello,

Ok, I will check it. Thank you for the report

noho501 commented 1 week ago

Hi @pedroSG94 ,

How is it going?

pedroSG94 commented 1 week ago

Hello,

Sorry for no report back. I was testing with 4 differents devices but I'm not able to reproduce it. The sync and audio is the same that at the start doing tests of 40min+ (quality is good)

Tested devices: Pixel 7, Pixel 6a, Xiaomi A3, Samsung A50

I was using the app example by default and adding this line:

setTimestampMode(TimestampMode.BUFFER, TimestampMode.BUFFER)

For now, I'm testing in a local server

Did you tried with a local server instead YouTube to discard that the problem only happens in YouTube? (I will check this now too) Can you tell me your video and audio configuration to do the test with the same values? Did you modify the volume of the microphone or internal? Did you tried remove TimestampMode.BUFFER to know if the delay problem is related with that? (You maybe will have pops on YouTube but you can discard the other problem)

noho501 commented 3 days ago

Hi @pedroSG94

I just tested to the latest version and noticed an improvement in the delayed sound compared to the previous version. However, the sound still becomes distorted when the microphone is turned on.

You can listen to an example of the issue in this live stream: [YouTube Link]

I set this line on my code: setTimestampMode(TimestampMode.BUFFER, TimestampMode.BUFFER)

Audio: 48Khz/128kps

Best regards,

pedroSG94 commented 2 days ago

Hello,

Thank you for the info. I will check it using that audio config. If you are using this to increase the volume of the microphone try to remove the line:

microphoneVolume = 2f

If the device audio is fine but only fail the microphone audio using MixAudioSource and it is working using MicrophoneSource, the problem could be related with the audio volumen filter or the audio mixer.