search

pedroSG94 / RootEncoder

RootEncoder for Android (rtmp-rtsp-stream-client-java) is a stream encoder to push video/audio to media servers using protocols RTMP, RTSP, SRT and UDP with all code written in Java/Kotlin
Apache License 2.0
2.57k stars 775 forks source link

Bitrate, keyframe intervals, and sample rate not stable #1588

Closed noho501 closed 1 month ago

noho501 commented 1 month ago

Hi @pedroSG94 ,

We have implemented a ScreenService class to stream to YouTube, but we are encountering some issues related to bitrate, keyframe intervals, and sample rate after streaming for a few minutes. Errors appear in YouTube Studio. Do you have any suggestions or insights on where the issue might be in our code? Below is the code for your reference.

4:44 PM
Error
Please use a keyframe frequency of four seconds or less. Currently, keyframes are not being sent often enough, which can cause buffering. The current keyframe frequency is 6.0 seconds. Note that ingestion errors can cause incorrect GOP (group of pictures) sizes.

4:44 PM
Warning
The stream's current bitrate (1619.86 Kbps) is lower than the recommended bitrate. We recommend that you use a stream bitrate of 6800 Kbps.

4:44 PM
Suggestion
The current sample rate is 0. The recommended sample rates are 44.1kHz and 48kHz.

4:42 PM
Error
Please use a keyframe frequency of four seconds or less. Currently, keyframes are not being sent often enough, which can cause buffering. The current keyframe frequency is 6.0 seconds. Note that ingestion errors can cause incorrect GOP (group of pictures) sizes.

4:42 PM
Warning
The stream's current bitrate (1720.95 Kbps) is lower than the recommended bitrate. We recommend that you use a stream bitrate of 6800 Kbps.

4:42 PM
Suggestion
The current sample rate is 0. The recommended sample rates are 44.1kHz and 48kHz.

4:40 PM
Warning
The stream's current bitrate (424.37 Kbps) is lower than the recommended bitrate. We recommend that you use a stream bitrate of 6800 Kbps.

4:38 PM
Stream is healthy
Stream health is excellent.

Thank you in advance for your help!

ScreenService.txt

pedroSG94 commented 1 month ago

Hello,

For now, try increasing the value of forcefps to 30 or the value set to prepareVideo.

This should solve the keyframe value because you will send a constant 30fps to the stream server producing the correct FPS key frame interval.

The bitrate depends on the screen that you are recording, if you have the same image all the time the bitrate is reduced because that is the way the video encoder works (that is part of the h264 algorithm to reduce video size). You can avoid it using an image that changes all the time, for example, open a video and checking it.

About the sample rate to 0 is weird. Try setting the value to 44100 as recommended by YouTube (you have 32000). Maybe that is because 32000 is not a recommended value but if you can hear the audio all should be fine.

Anyway, all that seems like warnings so meanwhile you can check that the result is fine you can ignore it.

noho501 commented 1 month ago

Hi @pedroSG94 ,

How can I set the forced FPS? I wasn’t able to find where to configure it. Could you please guide me?

pedroSG94 commented 1 month ago

This line

getGlInterface().setForceRender(true, 15)

To:

getGlInterface().setForceRender(true, fps)

noho501 commented 1 month ago

Thank you for your assistance. We will update our code and test it accordingly.

noho501 commented 1 month ago

Hi @pedroSG94 ,

Thank you for your suggestion. We tested on the Redmi 10 (2022) with Android version 12, and encountered an issue where the live stream video freezes, but the audio continues to process. Interestingly, the saved video has no issues—this problem only occurs during the live stream. Below is the log. Do you have any suggestions?

bufferpool2 0xb400007b78dfaa28 : 5(40960 size) total buffers - 1(8192 size) used buffers - 1558/1563 (recycle/alloc) - 14/3122 (fetch/transfer)
2024-09-23 11:13:22.807 10480-10660 RtmpSender           I  wrote Audio packet, size 386
2024-09-23 11:13:22.814 10480-10660 RtmpSender           I  wrote Audio packet, size 410
2024-09-23 11:13:22.815 10480-10660 RtmpSender           I  wrote Audio packet, size 399
2024-09-23 11:13:22.815 10480-10660 RtmpSender           I  wrote Audio packet, size 374
2024-09-23 11:13:22.819 10480-10660 RtmpSender           I  wrote Video packet, size 19591
2024-09-23 11:13:22.849 10480-10660 RtmpSender           I  wrote Video packet, size 16374
2024-09-23 11:13:22.863 10480-10660 RtmpSender           I  wrote Video packet, size 1928
2024-09-23 11:13:22.909 10480-10660 RtmpSender           I  wrote Video packet, size 16798
2024-09-23 11:13:22.920 10480-10660 RtmpSender           I  wrote Video packet, size 14554
2024-09-23 11:13:22.933 10480-10660 RtmpSender           I  wrote Video packet, size 2033
2024-09-23 11:13:22.957 10480-10660 RtmpSender           I  wrote Video packet, size 17898
2024-09-23 11:13:22.978 10480-10660 RtmpSender           I  wrote Video packet, size 17294
2024-09-23 11:13:22.989 10480-10660 RtmpSender           I  wrote Audio packet, size 361
2024-09-23 11:13:22.990 10480-10660 RtmpSender           I  wrote Audio packet, size 350
2024-09-23 11:13:23.012 10480-10660 RtmpSender           I  wrote Video packet, size 14510
2024-09-23 11:13:23.034 10480-10660 RtmpSender           I  wrote Audio packet, size 342
2024-09-23 11:13:23.038 10480-10660 RtmpSender           I  wrote Audio packet, size 385
2024-09-23 11:13:23.040 10480-10660 RtmpSender           I  wrote Audio packet, size 379
2024-09-23 11:13:23.041 10480-10660 RtmpSender           I  wrote Audio packet, size 405
2024-09-23 11:13:23.042 10480-10660 RtmpSender           I  wrote Audio packet, size 389
2024-09-23 11:13:23.043 10480-10660 RtmpSender           I  wrote Audio packet, size 391
2024-09-23 11:13:23.045 10480-10660 RtmpSender           I  wrote Audio packet, size 412
2024-09-23 11:13:23.045 10480-10660 RtmpSender           I  wrote Audio packet, size 382
reset()
2024-09-23 11:18:20.060 10480-10480 MPEG4Writer          D  Video track stopping. Stop source
2024-09-23 11:18:20.060 10480-10480 MPEG4Writer          D  Video track source stopping
2024-09-23 11:18:20.060 10480-10480 MPEG4Writer          D  Video track source stopped
2024-09-23 11:18:20.060 10480-13860 MediaWriter          V  Track event err/info msg:101, trackId:1, type:1000,val:-1011
2024-09-23 11:18:20.061 10480-13860 MPEG4Writer          I  Received total/0-length (1087/0) buffers and encoded 1087 frames. - Video
2024-09-23 11:18:20.061 10480-10480 MPEG4Writer          D  Video track stopped. Status:0. Stop source
2024-09-23 11:18:20.062 10480-10660 RtmpSender           I  wrote Video packet, size 159
2024-09-23 11:18:20.063 10480-13861 MediaWriter          V  Track event err/info msg:101, trackId:2, type:1000,val:-1011
2024-09-23 11:18:20.063 10480-13861 MPEG4Writer          I  Received total/0-length (1232/0) buffers and encoded 1232 frames. - Audio
2024-09-23 11:18:20.063 10480-13861 MPEG4Writer          I  Audio track drift time: 0 us
2024-09-23 11:18:20.063 10480-10480 MPEG4Writer          D  Audio track stopping. Stop source
2024-09-23 11:18:20.063 10480-10480 MPEG4Writer          D  Audio track source stopping
2024-09-23 11:18:20.063 10480-10480 MPEG4Writer          D  Audio track source stopped
2024-09-23 11:18:20.063 10480-10480 MPEG4Writer          D  Audio track stopped. Status:0. Stop source
2024-09-23 11:18:20.063 10480-10480 MPEG4Writer          D  Duration from tracks range is [28615499, 28741212] us
2024-09-23 11:18:20.063 10480-13858 MPEG4Writer          D  0 chunks are written in the last batch
2024-09-23 11:18:20.063 10480-10480 MPEG4Writer          D  WriterThread stopped. Status:0
2024-09-23 11:18:20.064 10480-10480 MPEG4Writer          I  Adjust the moov start time from 0 us -> 0 us
2024-09-23 11:18:20.064 10480-10480 MPEG4Writer          I  The mp4 file will not be streamable.
2024-09-23 11:18:20.064 10480-10480 MPEG4Writer          I  MOOV atom was written to the file
2024-09-23 11:18:20.064 10480-10480 MPEG4Writer          D  release()
2024-09-23 11:18:20.073 10480-10660 RtmpSender           I  wrote Video packet, size 555
2024-09-23 11:18:20.075 10480-10480 MPEG4Writer          D  final fsync() takes 11 ms, file size 16849793
2024-09-23 11:18:20.076 10480-10480 MPEG4Writer          D  MP4WtrCtrlHlpLooper stopped
2024-09-23 11:18:20.076 10480-10480 MPEG4Writer          D  Top 5 write durations(microseconds): #1:4128 #2:4804 #3:4936 #4:7021 #5:8223
2024-09-23 11:18:20.076 10480-10480 MPEG4Writer          D  reset()
2024-09-23 11:18:20.076 10480-10480 MPEG4Writer          D  Video track stopping. Stop source
2024-09-23 11:18:20.076 10480-10480 MPEG4Writer          E  Stop() called but track is not started or stopped
2024-09-23 11:18:20.076 10480-10480 MPEG4Writer          D  Audio track stopping. Stop source
2024-09-23 11:18:20.076 10480-10480 MPEG4Writer          E  Stop() called but track is not started or stopped
2024-09-23 11:18:20.076 10480-10480 May                  I  ScreenBroadcast onStatusChange STOPPED
2024-09-23 11:18:20.077 10480-10480 May                  I  ScreenBroadcast onUpdateUI serviceIsRunning() = true
2024-09-23 11:18:20.087 10480-11632 CommandsManager         com.app.test                  I  send Command(name='closeStream', transactionId=6, timeStamp=29, streamId=1, data=[AmfString value: closeStream, AmfNumber value: 6.0, AmfNull], bodySize=24)
2024-09-23 11:18:20.097 10480-10981 BufferQueueProducer     com.app.test                  I  [SurfaceTexture-1-10480-2](id:28f000000019,api:1,p:791,c:10480) disconnect: api 1
2024-09-23 11:18:20.098 10480-10981 libMEOW              D  meow new tls: 0xb400007b9a489b80
2024-09-23 11:18:20.098 10480-10981 libMEOW              D  applied 1 plugins for [com.app.test]:
2024-09-23 11:18:20.098 10480-10981 libMEOW              D    plugin 1: [libMEOW_gift.so]:
2024-09-23 11:18:20.100 10480-10480 AudioRecord          D  stop mSessionID=249
2024-09-23 11:18:20.100 10480-10480 AudioRecord          D  stop(59): 0xb400007b9a52be00, mActive:1
2024-09-23 11:18:20.164 10480-10480 AudioRecord          D  stop(59) done
2024-09-23 11:18:20.166 10480-10480 AudioRecord          D  stop mSessionID=249
2024-09-23 11:18:20.166 10480-10480 AudioRecord          D  stop(59): 0xb400007b9a52be00, mActive:0
2024-09-23 11:18:20.166 10480-10480 AudioRecord          D  stop mSessionID=249
2024-09-23 11:18:20.166 10480-10480 AudioRecord          D  stop(59): 0xb400007b9a52be00, mActive:0
2024-09-23 11:18:20.166 10480-10480 AudioRecord          D  ~AudioRecord(59): mStatus 0
2024-09-23 11:18:20.166 10480-10480 AudioRecord          D  stop mSessionID=249
2024-09-23 11:18:20.166 10480-10480 AudioRecord          D  stop(59): 0xb400007b9a52be00, mActive:0
2024-09-23 11:18:20.167 10480-10480 MicrophoneManager       com.app.test                  I  Microphone stopped
2024-09-23 11:18:20.174 10480-13838 BufferQueueProducer     com.app.test                  I  [GraphicBufferSource](id:28f000000018,api:1,p:10480,c:10480) disconnect: api 1
2024-09-23 11:18:20.176 10480-13838 GraphicBufferSource     com.app.test                  W  released unpopulated slots: [7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63]
2024-09-23 11:18:20.177 10480-13838 SurfaceManager          com.app.test                  I  GL released
2024-09-23 11:18:20.179 10480-13838 SurfaceManager          com.app.test                  I  GL released
2024-09-23 11:18:20.179 10480-13838 SurfaceManager          com.app.test                  E  GL already released
2024-09-23 11:18:20.181 10480-13838 SurfaceManager          com.app.test                  I  GL released
2024-09-23 11:18:20.181 10480-13838 libEGL               E  call to OpenGL ES API with no current context (logged once per thread)
2024-09-23 11:18:20.185 10480-13838 BufferQueueConsumer     com.app.test                  I  [SurfaceTexture-1-10480-2](id:28f000000019,api:0,p:-1,c:10480) disconnect
2024-09-23 11:18:20.186 10480-10480 VideoEncoder            com.app.test                  I  stopped
2024-09-23 11:18:20.189 10480-13838 libMEOW              D  meow delete tls: 0xb400007b782da100
2024-09-23 11:18:20.201 10480-13809 PipelineWatcher         com.app.test                  D  onWorkDone: frameIndex not found (1102); ignored
2024-09-23 11:18:20.210 10480-13809 BufferQueueConsumer     com.app.test                  I  [GraphicBufferSource](id:28f000000018,api:0,p:-1,c:10480) disconnect
2024-09-23 11:18:20.213 10480-10480 AudioEncoder            com.app.test                  I  stopped
2024-09-23 11:18:20.215 10480-13822 CCodecBuffers           com.app.test                  D  [c2.android.aac.encoder#972:1D-Output.Impl[N]] Client returned a buffer it does not own according to our record: 0
2024-09-23 11:18:20.215 10480-13822 CCodecBuffers           com.app.test                  D  [c2.android.aac.encoder#972:1D-Output.Impl[N]] Client returned a buffer it does not own according to our record: 1
2024-09-23 11:18:20.216 10480-13809 hw-BpHwBinder           com.app.test                  I  onLastStrongRef automatically unlinking death recipients
2024-09-23 11:18:20.218 10480-13808 MediaCodec           D  flushMediametrics
2024-09-23 11:18:20.220 10480-10480 VideoEncoder            com.app.test                  I  5 encoders found
2024-09-23 11:18:20.220 10480-10480 VideoEncoder            com.app.test                  I  Encoder c2.mtk.avc.encoder
2024-09-23 11:18:20.220 10480-10480 VideoEncoder            com.app.test                  I  Color supported: 2130708361
2024-09-23 11:18:20.220 10480-10480 VideoEncoder            com.app.test                  I  Encoder selected c2.mtk.avc.encoder
2024-09-23 11:18:20.221 10480-13808 MediaCodec           D  flushMediametrics
2024-09-23 11:18:20.224 10480-13822 hw-BpHwBinder           com.app.test                  I  onLastStrongRef automatically unlinking death recipients
2024-09-23 11:18:20.225 10480-14158 CCodec               D  allocate(c2.mtk.avc.encoder)
2024-09-23 11:18:20.225 10480-13822 MediaCodec           D  flushMediametrics
2024-09-23 11:18:20.226 10480-13822 MediaCodec           D  flushMediametrics
2024-09-23 11:18:20.229 10480-14158 CCodec               I  setting up 'default' as default (vendor) store
2024-09-23 11:18:20.249 10480-14158 CCodec               I  Created component [c2.mtk.avc.encoder]
2024-09-23 11:18:20.249 10480-14158 CCodecConfig            com.app.test                  D  read media type: video/avc
2024-09-23 11:18:20.255 10480-14158 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: algo.buffers.max-count.values
2024-09-23 11:18:20.256 10480-14158 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: output.subscribed-indices.values
2024-09-23 11:18:20.256 10480-14158 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: input.buffers.allocator-ids.values
2024-09-23 11:18:20.256 10480-14158 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: output.buffers.allocator-ids.values
2024-09-23 11:18:20.257 10480-14158 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: algo.buffers.allocator-ids.values
2024-09-23 11:18:20.257 10480-14158 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: output.buffers.pool-ids.values
2024-09-23 11:18:20.257 10480-14158 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: algo.buffers.pool-ids.values
2024-09-23 11:18:20.261 10480-14158 ReflectedParamUpdater   com.app.test                  D  ignored struct field coding.gop.values
2024-09-23 11:18:20.263 10480-14158 ReflectedParamUpdater   com.app.test                  D  ignored struct field coding.qp.values
2024-09-23 11:18:20.264 10480-14158 CCodecConfig            com.app.test                  D  ignoring local param raw.color (0xc2001809) as it is already supported
2024-09-23 11:18:20.265 10480-14158 CCodecConfig            com.app.test                  I  query failed after returning 22 values (BAD_INDEX)
2024-09-23 11:18:20.265 10480-14158 CCodecConfig            com.app.test                  D  c2 config diff is Dict {
                                                                                                      c2::u32 algo.bitrate-mode.value = 3
                                                                                                      c2::u32 algo.delay.value = 8
                                                                                                      c2::i32 algo.priority.value = 0
                                                                                                      c2::u32 algo.quality.value = 80
                                                                                                      c2::float algo.rate.value = 0
                                                                                                      c2::u32 coded.bitrate.value = 64000
                                                                                                      c2::float coded.frame-rate.value = 30
                                                                                                      c2::u32 coded.pl.level = 20494
                                                                                                      c2::u32 coded.pl.profile = 20484
                                                                                                      c2::u32 coded.tile-layout.columns = 2
                                                                                                      c2::u32 coded.tile-layout.order = 0
                                                                                                      c2::u32 coded.tile-layout.rows = 1
                                                                                                      c2::u32 coded.tile-layout.tile.height = 256
                                                                                                      c2::u32 coded.tile-layout.tile.width = 256
                                                                                                      c2::u32 coded.vui.color.matrix = 0
                                                                                                      c2::u32 coded.vui.color.primaries = 0
                                                                                                      c2::u32 coded.vui.color.range = 0
                                                                                                      c2::u32 coded.vui.color.transfer = 0
                                                                                                      c2::u32 coding.intra-refresh.mode = 0
                                                                                                      c2::float coding.intra-refresh.period = 0
                                                                                                      c2::u32 coding.request-sync-frame.value = 0
                                                                                                      c2::i64 coding.sync-frame-interval.value = 1000000
                                                                                                      Buffer coding.temporal-layering = {
                                                                                                        00000000:  10 00 00 00 07 20 01 52  00 00 00 00 00 00 00 00  ..... .R........
                                                                                                      }
                                                                                                      c2::u32 input.de
2024-09-23 11:18:20.265 10480-14158 ColorUtils           W  expected specified color aspects (0:0:0:0)
2024-09-23 11:18:20.267 10480-10480 VideoEncoder            com.app.test                  I  Prepare video info: SURFACE, 1080x1920
2024-09-23 11:18:20.267 10480-10480 VideoEncoder            com.app.test                  I  bitrate mode CBR not supported using default mode
2024-09-23 11:18:20.269 10480-14157 MediaCodec           I  MediaCodec will operate in async mode
2024-09-23 11:18:20.269 10480-10480 VQApply              D  minquality: requested QP unsupported, boost bitrate 4000000 by 800000
2024-09-23 11:18:20.269 10480-10480 VQApply              D  minquality/target bitrate raised from 4000000 to 4800000 bps
2024-09-23 11:18:20.270 10480-10480 MediaCodec           D  shapeMediaFormat: deltas(2): AMessage(what = 0x00000000) = {
                                                                                                        int32_t bitrate = 4800000
                                                                                                        int32_t android._encoding-quality-level = 0
                                                                                                      }
2024-09-23 11:18:20.270 10480-14157 MediaCodec           D  flushMediametrics
2024-09-23 11:18:20.270 10480-14158 CCodec               D  [c2.mtk.avc.encoder] buffers are bound to CCodec for this session
2024-09-23 11:18:20.271 10480-14158 CCodecConfig            com.app.test                  D  no c2 equivalents for color-format
2024-09-23 11:18:20.271 10480-14158 CCodecConfig            com.app.test                  D  no c2 equivalents for flags
2024-09-23 11:18:20.271 10480-14158 CCodecConfig            com.app.test                  D  no c2 equivalents for encoder
2024-09-23 11:18:20.272 10480-14158 CCodecConfig            com.app.test                  D  c2 config diff is   c2::u32 coded.bitrate.value = 4800000
                                                                                                      c2::float coded.frame-rate.value = 60
                                                                                                      c2::i64 coding.sync-frame-interval.value = 2000000
                                                                                                      c2::u32 raw.pixel-format.value = 34
                                                                                                      c2::u32 raw.size.height = 1920
                                                                                                      c2::u32 raw.size.width = 1080
2024-09-23 11:18:20.273 10480-14158 ColorUtils           W  expected specified color aspects (0:0:0:0)
2024-09-23 11:18:20.274 10480-14158 Codec2Client            com.app.test                  W  query -- param skipped: index = 3254781982.
2024-09-23 11:18:20.274 10480-14158 CCodec               D  setup formats input: AMessage(what = 0x00000000) = {
                                                                                                      int32_t android._color-format = 2130708361
                                                                                                      Rect crop(0, 0, 1079, 1919)
                                                                                                      int32_t color-standard = 0
                                                                                                      int32_t color-range = 0
                                                                                                      int32_t color-transfer = 0
                                                                                                      int32_t frame-rate = 60
                                                                                                      int32_t height = 1920
                                                                                                      float input.time-stretch = 1.000000
                                                                                                      int32_t intra-refresh-period = 0
                                                                                                      int32_t latency = 8
                                                                                                      string mime = "video/raw"
                                                                                                      int32_t prepend-sps-pps-to-idr-frames = 0
                                                                                                      int32_t priority = 0
                                                                                                      int32_t width = 1080
                                                                                                      int32_t android._dataspace = 0
                                                                                                      int64_t android._C2MemoryUsage = 0
                                                                                                      int32_t color-format = 2130708361
                                                                                                    }
2024-09-23 11:18:20.274 10480-14158 CCodec               D  setup formats output: AMessage(what = 0x00000000) = {
                                                                                                      int32_t bitrate = 4800000
                                                                                                      int32_t bitrate-mode = 1
                                                                                                      Rect crop(0, 0, 1079, 1919)
                                                                                                      int32_t width = 1080
                                                                                                      int32_t color-standard = 0
                                                                                                      int32_t color-range = 0
                                                                                                      int32_t color-transfer = 0
                                                                                                      int32_t frame-rate = 60
                                                                                                      int32_t height = 1920
                                                                                                      float input.time-stretch = 1.000000
                                                                                                      int32_t intra-refresh-period = 0
                                                                                                      int32_t latency = 8
                                                                                                      int32_t level = 16384
                                                                                                      int32_t max-bitrate = 4800000
                                                                                                      string mime = "video/avc"
                                                                                                      int32_t prepend-sps-pps-to-idr-frames = 0
                                                                                                      int32_t priority = 0
                                                                                                      int32_t profile = 8
                                                                                                    }
2024-09-23 11:18:20.274 10480-14158 CCodecConfig            com.app.test                  I  query failed after returning 23 values (BAD_INDEX)
2024-09-23 11:18:20.274 10480-14158 CCodecConfig            com.app.test                  D  c2 config diff is   c2::u32 algo.delay.value = 2
                                                                                                      c2::u32 coded.tile-layout.columns = 5
                                                                                                      c2::u32 coded.tile-layout.rows = 8
2024-09-23 11:18:20.275 10480-14158 ColorUtils           W  expected specified color aspects (0:0:0:0)
2024-09-23 11:18:20.275 10480-14158 BufferQueueConsumer     com.app.test                  I  [GraphicBufferSource](id:28f00000001b,api:0,p:-1,c:10480) connect: controlledByApp=false
2024-09-23 11:18:20.276 10480-14158 CCodec               D  input format changed to AMessage(what = 0x00000000) = {
                                                                                                      int32_t android._color-format = 2130708361
                                                                                                      Rect crop(0, 0, 1079, 1919)
                                                                                                      int32_t color-standard = 1
                                                                                                      int32_t color-range = 2
                                                                                                      int32_t color-transfer = 3
                                                                                                      int32_t frame-rate = 60
                                                                                                      int32_t height = 1920
                                                                                                      float input.time-stretch = 1.000000
                                                                                                      int32_t intra-refresh-period = 0
                                                                                                      int32_t latency = 2
                                                                                                      string mime = "video/raw"
                                                                                                      int32_t prepend-sps-pps-to-idr-frames = 0
                                                                                                      int32_t priority = 0
                                                                                                      int32_t width = 1080
                                                                                                      int32_t android._dataspace = 260
                                                                                                      int64_t android._C2MemoryUsage = 0
                                                                                                      int32_t color-format = 2130708361
                                                                                                    }
2024-09-23 11:18:20.276 10480-14158 GraphicBufferSource     com.app.test                  D  setting dataspace: 0x104, acquired=0
2024-09-23 11:18:20.277 10480-14158 CCodec               D  ISConfig not changed
2024-09-23 11:18:20.278 10480-10480 VideoEncoder            com.app.test                  I  prepared
2024-09-23 11:18:20.279 10480-10480 AudioEncoder            com.app.test                  I  2 encoders found
2024-09-23 11:18:20.280 10480-10480 AudioEncoder            com.app.test                  I  Encoder selected c2.android.aac.encoder
2024-09-23 11:18:20.282 10480-14168 CCodec               D  allocate(c2.android.aac.encoder)
2024-09-23 11:18:20.284 10480-14168 CCodec               I  setting up 'default' as default (vendor) store
2024-09-23 11:18:20.288 10480-14168 CCodec               I  Created component [c2.android.aac.encoder]
2024-09-23 11:18:20.292 10480-14168 CCodecConfig            com.app.test                  D  read media type: audio/mp4a-latm
2024-09-23 11:18:20.295 10480-14168 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: algo.buffers.max-count.values
2024-09-23 11:18:20.295 10480-14168 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: output.subscribed-indices.values
2024-09-23 11:18:20.295 10480-14168 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: input.buffers.allocator-ids.values
2024-09-23 11:18:20.295 10480-14168 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: output.buffers.allocator-ids.values
2024-09-23 11:18:20.296 10480-14168 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: algo.buffers.allocator-ids.values
2024-09-23 11:18:20.296 10480-14168 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: output.buffers.pool-ids.values
2024-09-23 11:18:20.296 10480-14168 ReflectedParamUpdater   com.app.test                  D  extent() != 1 for single value type: algo.buffers.pool-ids.values
2024-09-23 11:18:20.298 10480-14168 CCodecConfig            com.app.test                  I  query failed after returning 9 values (BAD_INDEX)
2024-09-23 11:18:20.298 10480-14168 CCodecConfig            com.app.test                  D  c2 config diff is Dict {
                                                                                                      c2::u32 coded.bitrate.value = 64000
                                                                                                      c2::u32 coded.pl.level = 0
                                                                                                      c2::u32 coded.pl.profile = 8192
                                                                                                      c2::u32 coding.aac-sbr-mode.value = 3
                                                                                                      c2::u32 input.buffers.max-size.value = 2048
                                                                                                      c2::u32 input.delay.value = 0
                                                                                                      string input.media-type.value = "audio/raw"
                                                                                                      string output.media-type.value = "audio/mp4a-latm"
                                                                                                      c2::u32 raw.channel-count.value = 1
                                                                                                      c2::u32 raw.sample-rate.value = 44100
                                                                                                    }
2024-09-23 11:18:20.299 10480-14168 MediaCodec           I  MediaCodec will operate in async mode
2024-09-23 11:18:20.300 10480-10480 MediaCodec           D  shapeMediaFormat: deltas(0): AMessage(what = 0x00000000) = {
                                                                                                      }
2024-09-23 11:18:20.300 10480-14168 MediaCodec           D  flushMediametrics
2024-09-23 11:18:20.300 10480-14168 CCodec               D  [c2.android.aac.encoder] buffers are bound to CCodec for this session
2024-09-23 11:18:20.300 10480-14168 CCodecConfig            com.app.test                  D  no c2 equivalents for flags
2024-09-23 11:18:20.300 10480-14168 CCodecConfig            com.app.test                  D  no c2 equivalents for encoder
2024-09-23 11:18:20.301 10480-14168 CCodecConfig            com.app.test                  D  c2 config diff is   c2::u32 coded.bitrate.value = 128000
                                                                                                      c2::u32 input.buffers.max-size.value = 4096
                                                                                                      c2::u32 raw.channel-count.value = 2
2024-09-23 11:18:20.301 10480-14168 Codec2Client            com.app.test                  W  query -- param skipped: index = 1107298332.
2024-09-23 11:18:20.301 10480-14168 CCodec               D  setup formats input: AMessage(what = 0x00000000) = {
                                                                                                      int32_t sample-rate = 44100
                                                                                                      int32_t channel-count = 2
                                                                                                      int32_t max-input-size = 8192
                                                                                                      string mime = "audio/raw"
                                                                                                    }
2024-09-23 11:18:20.301 10480-14168 CCodec               D  setup formats output: AMessage(what = 0x00000000) = {
                                                                                                      int32_t sample-rate = 44100
                                                                                                      int32_t bitrate = 128000
                                                                                                      int32_t channel-count = 2
                                                                                                      int32_t level = 0
                                                                                                      int32_t max-bitrate = 128000
                                                                                                      string mime = "audio/mp4a-latm"
                                                                                                      int32_t profile = 2
                                                                                                    }
2024-09-23 11:18:20.301 10480-14168 CCodecConfig            com.app.test                  I  query failed after returning 9 values (BAD_INDEX)
2024-09-23 11:18:20.302 10480-10480 AudioEncoder            com.app.test                  I  prepared
2024-09-23 11:18:20.334 10480-10480 May                  I  ScreenBroadcast onDisconnect
2024-09-23 11:18:20.334 10480-10480 May                  I  ScreenBroadcast onUpdateUI serviceIsRunning() = false
pedroSG94 commented 1 month ago

Hello,

Can you see video packets in the logcat when the stream is freezed? Did you test if this happens with Camera2Source?

noho501 commented 1 month ago

Hi @pedroSG94 ,

Above is the logcat during streaming. When the stream freezes, no errors appear, and we are screen broadcasting. Please note that the video saved locally works fine. Additionally, when I switch to another app (such as TikTok or the Photos app), the video initially shows the first frame and then freezes. We are streaming at 1080p60.

However, when streaming at 720p60, the video and audio become out of sync.

Did you test if this happens with Camera2Source? => Your mean live stream using the Camera?

Best regards,

pedroSG94 commented 1 month ago

Yes, instead of ScreenSource try with Camera2Source to discard problems with ScreenSource. Also, try use 30fps instead of 60fps.

However, when streaming at 720p60, the video and audio become out of sync.

This is weird, do you have this problem using other device?

noho501 commented 1 month ago

Yes, instead of ScreenSource try with Camera2Source to discard problems with ScreenSource. Also, try use 30fps instead of 60fps.

However, when streaming at 720p60, the video and audio become out of sync.

This is weird, do you have this problem using other device?

There are no issues with Oppo devices running Android 13 or 14. However, I’m experiencing problems on Redmi devices with Android 12.

pedroSG94 commented 1 month ago

Hello,

I did a change in the audio ts here: https://github.com/pedroSG94/RootEncoder/pull/1585/commits/77d71dbf380e81044590a0d8ce55dc32560cae1c

Compile this branch and test with screen example using your video/audio config to check if the problem is solved: https://github.com/pedroSG94/RootEncoder/pull/1585

noho501 commented 1 month ago

Hi,

Currently, I am unable to reproduce this error on the master branch or the fix/avoid-lock-flush branch. It seems to occur randomly, so I will temporarily close this issue. Once I am able to reproduce it consistently, I will let you know.

Thank you for your help!