Closed scottlamb closed 5 years ago
Here's the offending ffmpeg code, in libavformat/rtpdec.c
:
if (s->last_rtcp_ntp_time != AV_NOPTS_VALUE && s->ic->nb_streams > 1) {
int64_t addend;
int delta_timestamp;
/* compute pts from timestamp with received ntp_time */
delta_timestamp = timestamp - s->last_rtcp_timestamp;
/* convert to the PTS timebase */
addend = av_rescale(s->last_rtcp_ntp_time - s->first_rtcp_ntp_time,
s->st->time_base.den,
(uint64_t) s->st->time_base.num << 32);
pkt->pts = s->range_start_offset + s->rtcp_ts_offset + addend +
delta_timestamp;
return;
}
For others reading this issue, commit 091217b addressed the issue by isolating the video stream only thereby ignoring the stream marked "audio":
open_options.set(c_str!("allowed_media_types"), c_str!("video")).unwrap();
Further references:
FFmpeg Trac Ticket 5018 incorrect PTS/DTS on first frame of RTSP stream opened 11/20/2015 (as of 6/8/2020) Analyzed by developer: no
See also: FFmpeg Trac Ticket 6415 Strange DTS of first packet in RTSP stream
Note the ffmpeg ticket 5018 is a separate problem. Since it affects only the first frame, my workaround (in src/stream.rs
) is to just wait for the second key frame. It's annoying that it takes an extra second or two to open a stream but it's not a big problem.
if discard_first {
info!("Discarding the first packet to work around https://trac.ffmpeg.org/ticket/5018");
stream.get_next()?;
}
The discontinuity on SNTP adjustment is a bigger problem, not yet reported to ffmpeg folks; it's a deal-breaker for audio.
I'll review both issues and see if I can propose to you, Scott Lamb, a draft for submission to FFmpeg's Trac system. Can't do until this weekend.
It may be with the current high watermark of FFmpeg that the problem in FFmpeg Trac Ticket 5018 is less egregious. See my update: Comment #4
I decided it is appropriate to create a reproducible test with the least few variables and therefore sought a solution where: 1) an MP4 file is served by a locally controlled RTSP server. After spending a lot of time evaluating various RTSP server alternatives, I decided that project at https://github.com/aler9/rtsp-simple-server works well; they have downloadable binaries so you do not have to install a Go compiler. Here's what I established as a testing pattern.
Select a publicly available MP4 so everyone can use the same file.
3 Consoles.
Console A: launch the rts-simple-server
./rtsp-simple-server
Console B: stream with ffmpeg the file to the rtsp-simple-server
ffmpeg -re -stream_loop -1 -i row.mp4 -c copy -f rtsp rtsp://localhost:8554/mystream
Console C: pull the stream and run ffmpeg in debug mode. Preserve the color mode using "script"
Create a log file name for this test and launch script:
export TEST_LOG="ffmpeg_test_$(date +"%Y_%m_%d_%I_%M_%p").log"
script $TEST_LOG
then in the script shell:
ffmpeg -loglevel debug -rtsp_transport tcp -rtsp_flags prefer_tcp -t 240 -i rtsp://localhost:8554/mystream -map 0:v -filter:v showinfo -frames:v 3200 -f null /dev/nul
exit
then review the color-preserved log in the main shell:
cat -n $TEST_LOG | less
cat -n $TEST_LOG | grep error
The concern I have with the proposed test in Ffmpeg Trac Ticket 5018 (https://trac.ffmpeg.org/ticket/5018) is that
the rtsp server at wowzaec2demo.streamlock.net is an unknown quantity and may be a source of error.
The URL used in Ticket 5018 is an rtsp stream served up from the file BigBuckBunny_115k.mov. It looks like the file
"BigBuckBunny_115k.mov" is an alias created on the wowzaec2demo.streamlock.net.
The closest I could find of the video files is at http://bbb3d.renderfarming.net/download.html, specifically
http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_1080p_30fps_normal.mp4 which I
created a link to as "bunny.mp4". When I tried using FFmpeg to send bunny.mp4 to my
new rtsp server, FFmpeg errored out:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bunny.mp4': Metadata: major_brand : isom minor_version : 1 compatible_brands: isomavc1 creation_time : 2013-12-16T17:44:39.000000Z title : Big Buck Bunny, Sunflower version artist : Blender Foundation 2008, Janus Bager Kristensen 2013 comment : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net genre : Animation composer : Sacha Goedegebure Duration: 00:10:34.53, start: 0.000000, bitrate: 3481 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 2998 kb/s, 30 fps, 30 tbr, 30k tbn, 60 tbc (default) Metadata: creation_time : 2013-12-16T17:44:39.000000Z handler_name : GPAC ISO Video Handler Stream #0:1(und): Audio: mp3 (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 160 kb/s (default) Metadata: creation_time : 2013-12-16T17:44:42.000000Z handler_name : GPAC ISO Audio Handler Stream #0:2(und): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 320 kb/s (default) Metadata: creation_time : 2013-12-16T17:44:42.000000Z handler_name : GPAC ISO Audio Handler Side data: audio service type: main [rtp @ 0x561e2e20c6c0] Unsupported codec ac3 Could not write header for output file #0 (incorrect codec parameters ?): Operation not permitted Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:2 -> #0:1 (copy) Last message repeated 1 times
The ac3 codec was a proprietary codec developed by Dolby for Surroundsound.
Wikipedia tells us the patents finally expired 2017, but I suspect Gentoo carefully excluded it from its build of FFmpeg
in order respect the patent. Any rate, when I got into this ac3 rabbit hole, I decided to step back and see
if the initial test Scott submitted to FFmpeg 5 years ago still was a problem. It doesn't look like it is; Scott
may want to try again with a newer version of FFmpeg to see if he can duplicate the results. Scott has recently
indicated that this problem may no longer be a showstopper for audio, so I'll let my contribution above be
my last to this Issue. I do like my testing methodology that preserves the colorized output.
I also should add, I tried using FFmpeg against my Reolink cameras several weeks ago and duplicated the "monotonic" errors. Seeing the errors motivated me to help work on this issue. My version of FFmpeg has been upgraded since then, upgrades are a common thing when running Gentoo Linux, and today's attempt (7/3/2020) did not produce any "monotonic" errors.
jlpoole@taurus /tmp/test $ ffmpeg -loglevel debug -rtsp_transport tcp -rtsp_flags prefer_tcp -t 20 -i rtsp://admin:[REDACTED]@192.168.1.48:554/h264Preview_01_main -map 0:v -filter:v showinfo -frames:v 1200 -f null /dev/null ffmpeg version 4.3 Copyright (c) 2000-2020 the FFmpeg developers built with gcc 8.3.0 (Gentoo 8.3.0-r1 p1.1) configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-4.3/html --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --nm=x86_64-pc-linux-gnu-nm --ranlib=x86_64-pc-linux-gnu-ranlib --optflags='-march=native -O2 -pipe' --disable-static --enable-avfilter --enable-avresample --disable-stripping --disable-optimizations --disable-libcelt --disable-indev=alsa --disable-indev=oss --disable-indev=jack --disable-outdev=alsa --disable-outdev=oss --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --enable-gnutls --disable-gmp --enable-gpl --disable-hardcoded-tables --enable-iconv --disable-libtls --disable-libxml2 --disable-lzma --enable-network --disable-opencl --disable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl2 --disable-vaapi --disable-vdpau --disable-vulkan --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --disable-opengl --disable-libv4l2 --enable-libpulse --disable-libdrm --disable-libjack --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libcodec2 --enable-libdav1d --disable-libfdk-aac --disable-libopenjpeg --disable-libbluray --disable-libgme --disable-libgsm --disable-libaribb24 --disable-mmal --disable-libmodplug --enable-libopus --disable-libilbc --disable-librtmp --disable-libssh --disable-libspeex --disable-libsrt --disable-librsvg --disable-ffnvcodec --disable-libvorbis --disable-libvpx --disable-libzvbi --disable-appkit --disable-libbs2b --disable-chromaprint --disable-cuda-llvm --disable-libflite --disable-frei0r --disable-libfribidi --disable-fontconfig --disable-ladspa --disable-libass --disable-libtesseract --disable-lv2 --disable-libfreetype --disable-libvidstab --disable-librubberband --disable-libzmq --disable-libzimg --disable-libsoxr --enable-pthreads --disable-libvo-amrwbenc --disable-libmp3lame --disable-libkvazaar --enable-libaom --enable-libopenh264 --disable-libsnappy --disable-libtheora --disable-libtwolame --disable-libwavpack --disable-libwebp --enable-libx264 --enable-libx265 --disable-libxvid --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-altivec --disable-vsx --disable-power8 --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop --cpu=host --disable-doc --disable-htmlpages --enable-manpages libavutil 56. 51.100 / 56. 51.100 libavcodec 58. 91.100 / 58. 91.100 jlpoole@taurus /tmp/test $ cat -n $TEST_LOG|grep mono 121 Stream #0:1, 18, 1/16000: Audio: aac (LC), 16000 Hz, mono, fltp jlpoole@taurus /tmp/test $ cat -n $TEST_LOG|grep error 2013 603 frames successfully decoded, 0 decoding errors jlpoole@taurus /tmp/test $
There's an ffmpeg bug when a camera has audio enabled. I'll quote the troubleshooting guide below. Either we need this ffmpeg bug fixed (and the fixed version bundled with Moonfire NVR) or we need to switch RTSP libraries.
Error: pts not monotonically increasing; got 26615520 then 26539470
If your streams cut out with an error message like this one, there are a couple possibilities.
One is that your camera outputs B frames. If you believe this is the case, file a feature request; Moonfire NVR currently doesn't support B frames. You may be able to configure your camera to disable B frames in the meantime.
A more subtle problem occurs in cameras such as the Dahua Starlight series when the following is true:
Moonfire NVR currently uses the ffmpeg library to talk to the cameras. ffmpeg doesn't properly support this situation. It uses the NTP time to adjust the PTS and DTS, and thus experiences jumps forward and backward. The forward jumps cause one frame to be artificially lengthened. The backward jumps create an impossible situation which causes Moonfire NVR to abort the session and retry.
In the long term, Moonfire NVR will likely implement its own RTSP support.
In the short term, you can use either of two workarounds: