Use FFmpeg to detect audio, find when the silence start and when the silence end. But find issue: Application provided invalid, non monotonically increasing dts to muxer in stream 0: 247 >= 247

[JamesChungZLL]

• [ ] I have read the FAQ.
• [ ] I have searched in existing issues.

Environment

• OS: [macOS...]
• scrcpy version: [2.4]
• installation method: [brew]
• device model: vivo S10
• Android version: [13]

Describe the bug Use FFmpeg to detect audio, find when the silence start and when the silence end. But find issue: Application provided invalid, non monotonically increasing dts to muxer in stream 0: 247 >= 247

command: ffmpeg -i 1.mp4 -af silencedetect=noise=-30dB:d=0.05 -f null -

[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 247 >= 247
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 272 >= 272
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 276 >= 276
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 278 >= 278
[silencedetect @ 0x600001e9c120] silence_end: 4.88571 | silence_duration: 4.62021
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 282 >= 282
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 284 >= 284
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 287 >= 287
[silencedetect @ 0x600001e9c120] silence_start: 4.9515
[silencedetect @ 0x600001e9c120] silence_end: 5.0079 | silence_duration: 0.0563958
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 289 >= 289
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 291 >= 291
[silencedetect @ 0x600001e9c120] silence_start: 5.09154
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 299 >= 299
[silencedetect @ 0x600001e9c120] silence_end: 5.19271 | silence_duration: 0.101167
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 302 >= 302
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 304 >= 304
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 306 >= 306
[silencedetect @ 0x600001e9c120] silence_start: 5.23865
[null @ 0x11ee115b0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 309 >= 309

Please do not post screenshots of your terminal, just post the content as text instead.

[rom1v]

Thank you for your report.

I can reproduce if I record with both video and audio:

scrcpy --record=file.mp4
ffmpeg -i file.mp4 -af silencedetect=noise=-30dB:d=0.05 -f null -

However, if I record with audio only, there are no warnings:

scrcpy --record=file.mp4 --no-video
ffmpeg -i file.mp4 -af silencedetect=noise=-30dB:d=0.05 -f null -

Similarly, if I record to mkv with both video and audio, there are no warnings:

scrcpy --record=file.mkv
ffmpeg -i file.mkv -af silencedetect=noise=-30dB:d=0.05 -f null -

(That does not mean that the problem does not occur with mkv, maybe the ffmpeg demuxer just does not warn on such error.)

I suspect this is due to video/audio interleaving. If I log the pts and dts here:

diff --git a/app/src/recorder.c b/app/src/recorder.c
index 9e0b33959..12c501697 100644
--- a/app/src/recorder.c
+++ b/app/src/recorder.c
@@ -104,7 +104,10 @@ static bool
 sc_recorder_write_stream(struct sc_recorder *recorder,
                          struct sc_recorder_stream *st, AVPacket *packet) {
     AVStream *stream = recorder->ctx->streams[st->index];
+    int64_t pts = packet->pts;
     sc_recorder_rescale_packet(stream, packet);
+    LOGD("==== [%d] pts=%" PRIi64 "\trescaled(%d/%d)=%" PRIi64, st->index, pts,
+         stream->time_base.num, stream->time_base.den, packet->pts);
     if (st->last_pts != AV_NOPTS_VALUE && packet->pts <= st->last_pts) {
         LOGD("Fixing PTS non monotonically increasing in stream %d "
              "(%" PRIi64 " >= %" PRIi64 ")",

DEBUG: ==== [0] pts=2822636 rescaled(1/90000)=254037
DEBUG: ==== [1] pts=2856520 rescaled(1/48000)=137113
DEBUG: ==== [0] pts=2831304 rescaled(1/90000)=254817
DEBUG: ==== [0] pts=2839712 rescaled(1/90000)=255574
DEBUG: ==== [0] pts=2847253 rescaled(1/90000)=256253
DEBUG: ==== [1] pts=2876520 rescaled(1/48000)=138073
DEBUG: ==== [0] pts=2856169 rescaled(1/90000)=257055
DEBUG: ==== [0] pts=2864154 rescaled(1/90000)=257774
DEBUG: ==== [1] pts=2896520 rescaled(1/48000)=139033
DEBUG: ==== [0] pts=2873831 rescaled(1/90000)=258645
DEBUG: ==== [0] pts=2882748 rescaled(1/90000)=259447
DEBUG: ==== [1] pts=2916520 rescaled(1/48000)=139993
DEBUG: ==== [0] pts=2890764 rescaled(1/90000)=260169
DEBUG: ==== [0] pts=2898365 rescaled(1/90000)=260853

(0 is video, 1 is audio, only the pts values are logged, but the dts values are the same)

But the call to av_interleaved_write_packet() is expected to write the packets in the correct order:

https://github.com/Genymobile/scrcpy/blob/206809a99affad9a7aa58fcf7593cea71f48954d/app/src/recorder.c#L117

To be investigated…

[rom1v]

Btw, just remuxing/reencoding the stream does not yield any warnings.

I have no time to look at it for now, but could you please test with other filters in FFmpeg? Since this message comes from a muxer (see libavformat/mux.c), it might be related to this specific silencedetect filter.

[JamesChungZLL]

Btw, just remuxing/reencoding the stream does not yield any warnings.

I have no time to look at it for now, but could you please test with other filters in FFmpeg? Since this message comes from a muxer (see libavformat/mux.c), it might be related to this specific silencedetect filter.

Thanks for your reply~! No worries, I am not in the hurry for the fix. Just some warnings, still can detect the silence correctlly. I am now putting some energy into the video/audio area. Hoping I can fix issues by myself sometime. ^_^