Video & Audio frame discarded

[noho501]

Hi @pedroSG94 ,

I'm experiencing an issue where the stream stops automatically after around 15 minutes, and sometimes after 30-60 minutes. I’ve attached the log below for reference. Do you know the potential cause and how to resolve this?

Test device:

• Redmi 10 (2022)
• Android 12
• "Unlimited battery" enabled for the app

Test environment:

• Screen broadcasting (open TikTok app)
• Streaming on Restream.io

Any help would be appreciated!

stream-stop.txt

[pedroSG94]

This is because the sender coroutine is blocked sending a packet or death. This is really weird. Do you have this error only with the readmi?

[pedroSG94]

I did a new branch with fews modifications that can help with the case: https://github.com/pedroSG94/RootEncoder/pull/1593

Try that branch and let me know the result. This change also add info in the discard message to know if the coroutine is death

[noho501]

Thank you. I will try it. This error appear on some devices not only redmi.

[pedroSG94]

Looking in more detail the logs, the coroutine is not death so the problem is related with the coroutine that take so much time to send a packet. Sometimes take like 1s to send a packet. Make sure that the internet in your device is stable, with a good speed to use that bitrate and try with a lower bitrate. I will try to reproduce it and report you back

[noho501]

Hi @pedroSG94 ,

Do you know why I'm unable to use a snapshot version from Jitpack? I usually use it to test your library in our app, but now I'm getting the following error:

Failed to resolve: com.github.pedroSG94.RootEncoder:library:feature~socket-write-timeout-98bec9d80e-1

Here’s the link: https://www.jitpack.io/#pedroSG94/RootEncoder/feature~socket-write-timeout-98bec9d80e-1

[pedroSG94]

Try directly with the commit: 98bec9d80e

[pedroSG94]

implementation 'com.github.pedroSG94.RootEncoder:library:98bec9d80e'

[noho501]

Yes, I’ve got it now. I will test it and share the results with you. Thank you!

[noho501]

Hi @pedroSG94 ,

Here is log, please see it

stream-stop-2.txt

[pedroSG94]

Ok, that is not working. I was doing tests with a really high bitrate (30mbps) and the problem seems to be related with the bandwidth, at least in my case, because using version 2.5.1 (before ktor socket implementation) I have the same problem using the same bitrate but using 20mbps all is working fine in both cases.

I did an optimization to the TCP socket to reduce the calls to the socket. Also, I added the adaptative bitrate to handle bandwidth problems to the rotation example. Can you test using master branch with the rotation example? If you are using other example you can try add this adaptative bitrate like this: https://github.com/pedroSG94/RootEncoder/commit/67da0bb605beeb886d8821bc671e50e83637b800

With this change, you never should see frame discarded because the bitrate will change in real time depend of your bandwidth.

[noho501]

Hi,

"I’ll give it a try. Could you please create a snapshot on the master branch or a specific commit? I'm unable to retrieve master-52c31f98e6-1.

Thanks

[pedroSG94]

Try this gradle:

implementation 'com.github.pedroSG94.RootEncoder:library:5cb9677cfb'

Remember add the adaptative bitrate as indicated in the previous post

[noho501]

Hi @pedroSG94

I'm still seeing discarded logs. Additionally, after implementing adaptive bitrate, the stream on TikTok has a long delay of about 40-60 seconds (whereas YouTube has less delay and lag). The audio and video are also out of sync. Below is the log and the code I am using.

private val bitrateAdapter by lazy {
        BitrateAdapter {
            val adaptiveBitrate = SettingsUtils.adaptiveBitrate(this)
            if (adaptiveBitrate == 0) return@BitrateAdapter

            genericStream.setVideoBitrateOnFly(it)
        }.apply {
            setMaxBitrate(vBitrateHd + 128 * 1000)
        }
}

bitrateAdapter.setMaxBitrate(vBitrate + audioConfig.bitRate)

override fun onNewBitrate(bitrate: Long) {
        bitrateAdapter.adaptBitrate(bitrate, genericStream.getStreamClient().hasCongestion())
        callback?.onNewBitrate(bitrate)
}

Catlog: log-adapter-bitrate.txt

Video: https://drive.google.com/file/d/1jWgHuv3ImngZ4RDqJLXc00bZ0co2b4VN/view?usp=sharing

[pedroSG94]

Hello,

Ok, at least we can confirm that the problem is related with the bandwidth, because after stabilize the bitrate, discard logs disappear.

Lets try using a lower bitrate on prepareVideo and set the desired bitrate in bitrateAdapter.setMaxBitrate.

For example:

//bitrate start with 1mbps
prepareVideo(1280,  720, 1000 * 1000)
//bitrate increase gradually to 4mbps depend of the bandwidth
bitrateAdapter.setMaxBitrate(4000 * 1000)

This should remove totally the discard logs and the delay. Can you tell me your original video config? (resolution, fps and bitrate)

[noho501]

Here is the video configuration I'm using:

• Resolution: 1080x1920
• Frame Rate: 60fps
• Bitrate: 4000

I'll try your suggestion and update you with the results.

[noho501]

Hi again,

I’ve confirmed that the following settings work fine without any delay:

• Resolution: 720x1280
• Frame Rate: 30fps
• Bitrate: 1000 * 1000
• Max Adaptive Bitrate: 4000 * 1000

My question is, is there a way to automatically adjust the quality, frame rate, and bitrate based on the phone’s hardware? Users might not know how to configure the settings correctly, so it would be helpful to adjust them automatically. Do you have any suggestions?

[pedroSG94]

Hello,

No, I haven't a way to do it. You can try to do 3 config yourself (low, medium and high) and let the user select one. Using by default the medium. This way, the user should be able to change it easy if have problems with the default config.

[noho501]

Thanks for your help.