Closed jlpoolen closed 3 years ago
Well, looks like the panic in #113 is no longer happening, so that's an improvement at least.
Do you still have any logs from the older, working installation? I'd like to see the lines after Initialized ffmpeg. Versions:
if so.
It's not happy with garage_west
or peck_west
from the beginning. Is there anything different between these cameras and the others? Different model, firmware, etc?
I see several kinds of errors here:
W20210310 13:06:38.690 s-garage_west-main moonfire_ffmpeg::rtsp] 0xaf300950: CSeq 8 expected, 0 received.
W20210310 13:17:09.060 s-garage_west-main moonfire_ffmpeg::rtsp] 0xaf300950: RTP H.264 NAL unit type 29
W20210310 13:17:09.060 s-garage_west-main moonfire_ffmpeg::rtsp] 0xaf300950: is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
W20210310 14:00:48.748 s-garage_west-main moonfire_nvr::streamer] garage_west-main: sleeping for Duration { secs: 1, nanos: 0 } after error: pts not monotonically increasing; got 26140140 then 26120517
E20210310 14:22:37.305 s-garage_west-main moonfire_ffmpeg::rtsp] 0xac5ac8d0: RTP: PT=60: bad cseq f304 expected=3882
E20210310 14:35:12.457 s-garage_west-main moonfire_ffmpeg::h264] 0xac6598f0: Invalid NAL unit 8, skipping.
E20210310 14:22:37.361 s-garage_west-main moonfire_ffmpeg::h264] 0xac518460: error while decoding MB 22 47, bytestream -9
I20210310 14:22:37.373 s-garage_west-main moonfire_ffmpeg::h264] 0xac518460: concealing 6907 DC, 6907 AC, 6907 MV errors in I frame
The pts not monotonically increasing
is the direct cause of the disconnects, but I wonder if the other errors are due to the same root cause.
I wonder if you can reproduce with the ffmpeg
commandline utility, without involving Moonfire NVR at all. Knowing that would narrow the problem down anyway. Maybe via a command like this:
$ ffmpeg -report -loglevel debug -fflags nobuffer -rtsp_transport tcp -allowed_media_types video -i 'rtsp://admin:YOUR_PASSWORD_HERE@192.168.1.48:554/h264Preview_01_main' -c copy -f null -
That command will just open a stream and discard the received bytes until it gets dropped or you hit q
. Note you'll have to manually redact your password from the logfile.
If that shows problems, it'd be interesting to see if dropping the -fflags nobuffer
argument fixes it. Moonfire NVR started setting this option recently, in 75dce88.
We could also try getting a packet capture at the same time as a log; it'd be relatively easy then to see if ffmpeg is right about the CSeq
being incorrect. But it'd be even more data to deal with and harder to redact the password in so maybe we'll hold onto that idea for later.
Most of these complaints happen to a lesser extent on the other cameras too.
$ fgrep 'CSeq' logs.txt | cut -d' ' -f3 | sort | uniq -c
1016 s-garage_east-main
18265 s-garage_west-main
716 s-peck_east-main
243 s-peck_west-main
$ fgrep 'H.264 NAL unit' logs.txt | cut -d' ' -f3 | sort | uniq -c
2 s-garage_west-main
$ fgrep 'sleeping for Duration' logs.txt | cut -d' ' -f3 | sort | uniq -c
1 s-garage_east-main
164 s-garage_west-main
10 s-peck_east-main
3 s-peck_west-main
$ fgrep 'bad cseq' logs.txt | cut -d' ' -f3 | sort | uniq -c
1986452 s-garage_west-main
2895 s-peck_east-main
$ fgrep 'Invalid NAL' logs.txt | cut -d' ' -f3 | sort | uniq -c
17 s-garage_west-main
$ fgrep 'error while decoding MB' logs.txt | cut -d' ' -f3 | sort | uniq -c
1795 s-garage_west-main
58 s-peck_east-main
$ fgrep 'concealing' logs.txt | cut -d' ' -f3 | sort | uniq -c
5070 s-garage_west-main
125 s-peck_east-main
Further breakdown of the sleeping for Duration
lines:
$ fgrep 'pts not monotonically increasing' sleeps.txt | cut -d' ' -f3 | sort | uniq -c
1 s-garage_east-main
110 s-garage_west-main
4 s-peck_east-main
3 s-peck_west-main
$ fgrep 'excessive pts jump' sleeps.txt | cut -d' ' -f3 | sort | uniq -c
8 s-garage_west-main
4 s-peck_east-main
$ fgrep 'exceeds maximum' sleeps.txt | cut -d' ' -f3 | sort | uniq -c
28 s-garage_west-main
2 s-peck_east-main
$ fgrep 'Connection refused' sleeps.txt | cut -d' ' -f3 | sort | uniq -c
17 s-garage_west-main
$ fgrep 'Connection timed out' sleeps.txt | cut -d' ' -f3 | sort | uniq -c
1 s-garage_west-main
I'll see if I have some old logs... I doubt it, though. Also, I did upgrade the Raspberry Pi system, so I would think that I'm using a later version of ffmpeg. It looks like an upgrade of ffmpeg may be the culprit here. Nonetheless, I try to behave on the Raspberry Pi 4 server as if I were a consumer because that is the prospective user you may have for this project (if you are going to continue to suggest Raspberry Pi as a viable platform) and if they run into trouble doing an upgrade on the system, then it's good to flush the issue out with me. It's going to take some time which I'll try to devote over the weekend.
Good luck!
fwiw, I think a "consumer" would be more likely to follow the recommended path of using the Docker image, which comes with a fixed version of the ffmpeg libraries. But I don't know for sure yet if that would avoid this problem, or if it's a problem with ffmpeg at all. It's probably not the whole story; I have a pi4 with those same versions and that ffmpeg
command doesn't seem to be having the same problems with my cameras (in the first few minutes at least).
Hmm, in issue #84 you pasted a log including this:
I0710 190739.619 main moonfire_ffmpeg] Initialized ffmpeg. Versions:
avutil: running=56.22.100 compiled=56.22.100
avcodec: running=58.35.100 compiled=58.35.100
avformat: running=58.20.100 compiled=58.20.100
which is the same as the one from this bug:
I20210310 13:04:59.019 main moonfire_ffmpeg] Initialized ffmpeg. Versions:
avutil: running=56.22.100 compiled=56.22.100
avcodec: running=58.35.100 compiled=58.35.100
avformat: running=58.20.100 compiled=58.20.100
There could be some subtle difference not captured in the version number but it seems unlikely.
I found a log file that has: I0527 050128.310 main moonfire_nvr::cmds::run] Database is loaded. I0527 050128.373 main moonfire_nvr::cmds::run] Directories are opened. I0527 050128.373 main moonfire_nvr::cmds::run] Resolved timezone: America/Los_Angeles
W0527 050128.373 main moonfire_nvr::web] UI directory file "jquery-ui.bundle.js.gz" has unknown extension; skipping
W0527 050128.373 main moonfire_nvr::web] UI directory file "vendor.bundle.js.gz" has unknown extension; skipping
W0527 050128.373 main moonfire_nvr::web] UI directory file "jquery.bundle.js.gz" has unknown extension; skipping
W0527 050128.373 main moonfire_nvr::web] UI directory file "nvr.bundle.js.gz" has unknown extension; skipping
I0527 050128.374 main moonfire_ffmpeg] Initialized ffmpeg. Versions:
avutil: running=56.31.100 compiled=56.31.100
avcodec: running=58.54.100 compiled=58.54.100
avformat: running=58.29.100 compiled=58.29.100
Moreover, I cannot say for certain that in June or July I was using the same ffmpeg, but here's what is showing my Rasbertty Pi 4 now. I am not familiar with the Raspberry Pi deployment of binaries to know whether the July 8th date is the date the binary was installed on my machine, or an artificial date create by a some package manager.
jlpoole@raspberrypi:~ $ which ffmpeg
/usr/bin/ffmpeg
jlpoole@raspberrypi:~ $ ls -la /usr/bin/ffmpeg
-rwxr-xr-x 1 root root 243304 Jul 8 2020 /usr/bin/ffmpeg
jlpoole@raspberrypi:~ $ffmpeg -version
ffmpeg version 4.1.6-1~deb10u1+rpt1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --libdir=/usr/lib/arm-linux-gnueabihf --cpu=arm1176jzf-s --arch=arm
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
jlpoole@raspberrypi:~ $
I think what I need to do is just isolate the ffmpeg command for camera garage_west in its own session and see if I can recreate the error reports.
So I tried the following:
I connect to the Rapsberry Pi and with screen attempted to create four windows, each being a session using the following formula:
ffmpeg -loglevel debug -fflags nobuffer -rtsp_transport tcp -i 'rtsp://admin:[redacted]@192.168.1.49:554/h264Preview_01_main' -c copy -f null -
As can happen with screen, I became lost in its maze after trying to create a four panel window and unsuccessfully trying to give each a title. (Will still try with tmux another time.)
So I connected to the Raspberry Pi with four ssh consoles via Putty on Windows and ran ffmpeg against each camera. After about an hour, one then a second camera started to go into an infinite loop state with messages such as:
frame=442457 fps= 12 q=-1.0 size=N/A time=10:01:36.15 bitrate=N/A speed= 1x
Again, I cannot say if my ffmpeg on the Raspberry Pi 4 was altered when I performed a system upgrade in the last few months. But I suspected my disparity of results, e.g. June - January vs. recent few weeks, was being caused at the ffmpeg level.
I then wanted to compare my Gentoo VM version of ffmpeg with the Rapsberry Pi high watermark. I let the two consoles run overnight and 8 hours later found the Gentoo one in the endless loop "frame=...." above; the Raspberry Pi only had a handful of errors.
Gentoo ffmpeg
ares /home/jlpoole # ffmpeg -version
ffmpeg version 4.3.2 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 9.3.0 (Gentoo 9.3.0-r2 p4)
configuration: --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-4.3.2/html --mandir=/usr/share/man --enable-shared --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --nm=x86_64-pc-linux-gnu-nm --ranlib=x86_64-pc-linux-gnu-ranlib --pkg-config=x86_64-pc-linux-gnu-pkg-config --optflags='-march=native -O2 -pipe' --disable-static --enable-avfilter --enable-avresample --disable-stripping --disable-optimizations --disable-libcelt --enable-version3 --disable-indev=alsa --disable-indev=oss --disable-indev=jack --disable-indev=sndio --disable-outdev=alsa --disable-outdev=oss --disable-outdev=sndio --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --enable-gnutls --disable-gmp --enable-gpl --disable-hardcoded-tables --enable-iconv --disable-libtls --disable-libxml2 --disable-lzma --enable-network --disable-opencl --disable-openssl --enable-postproc --enable-libsmbclient --disable-ffplay --disable-sdl2 --disable-vaapi --disable-vdpau --disable-vulkan --enable-xlib --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --enable-opengl --disable-libv4l2 --enable-libpulse --disable-libdrm --disable-libjack --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libcodec2 --enable-libdav1d --disable-libfdk-aac --disable-libopenjpeg --disable-libbluray --disable-libgme --disable-libgsm --disable-libaribb24 --disable-mmal --disable-libmodplug --disable-libopus --disable-libilbc --disable-librtmp --disable-libssh --disable-libspeex --disable-libsrt --enable-librsvg --disable-ffnvcodec --enable-libvorbis --disable-libvpx --disable-libzvbi --disable-appkit --disable-libbs2b --disable-chromaprint --disable-cuda-llvm --disable-libflite --disable-frei0r --disable-libfribidi --disable-fontconfig --disable-ladspa --disable-libass --disable-libtesseract --disable-lv2 --enable-libfreetype --disable-libvidstab --disable-librubberband --disable-libzmq --disable-libzimg --disable-libsoxr --enable-pthreads --disable-libvo-amrwbenc --enable-libmp3lame --disable-libkvazaar --disable-libaom --disable-libopenh264 --disable-librav1e --disable-libsnappy --disable-libtheora --disable-libtwolame --disable-libwebp --enable-libx264 --disable-libx265 --disable-libxvid --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-mipsdsp --disable-mipsdspr2 --disable-mipsfpu --disable-altivec --disable-vsx --disable-power8 --disable-amd3dnow --disable-amd3dnowext --disable-aesni --disable-avx --disable-avx2 --disable-fma3 --disable-fma4 --disable-sse3 --disable-ssse3 --disable-sse4 --disable-sse42 --disable-xop --cpu=host --disable-doc --disable-htmlpages --enable-manpages
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
ares /home/jlpoole #
Raspberry Pi4 ffmpeg
jlpoole@raspberrypi:~ $ ffmpeg -version
ffmpeg version 4.1.6-1~deb10u1+rpt1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --libdir=/usr/lib/arm-linux-gnueabihf --cpu=arm1176jzf-s --arch=arm
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
jlpoole@raspberrypi:~ $
Note, that the Gentoo [server: ares] version does not have the library warning -- one of the benefits of compiling everything in Gentoo, methinks. It is unclear from Google searches what the risk of the library warning presents.
My next step is to "unmask" a more recent version of ffmpeg on my Gentoo VM and see what happens with that.
This issue is moving away from Moonfire-nvr and looking to be an ffmpeg issue.
Further thoughts: I suppose I should try to save what comes in via ffmeg's rtsp call to learn if the repeated message of "frame=442457 fps= 12 q=-1.0 size..." means nothing and affects what is processed, or whether it is white noise. I'm going to have to learn more about ffmpeg which is a monstrous tome, nay operating system guised as a program.
the Raspberry Pi only had a handful of errors.
What were the errors? I don't see them here. I'm particularly interested to see if it says similar errors to as ones you saw when running through Moonfire NVR:
CSeq 8 expected, 0 received.
RTP H.264 NAL unit type 29 is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
RTP: PT=60: bad cseq f304 expected=3882
Invalid NAL unit 8, skipping.
error while decoding MB 22 47, bytestream -9
concealing 6907 DC, 6907 AC, 6907 MV errors in I frame
I also don't know what you mean by "library warning". The ffmpeg -version
output from both looks normal to me.
The repeated frame=442457 fps= 12 q=-1.0 size...
message is normal; that's its way of showing progress. It shouldn't take an hour to start seeing this, though; I'd expect it to happen immediately.
Here's a copy from the Raspberry Pi console. See on line 5 "WARNING: library configuration mismatch".
jlpoole@raspberrypi:~ $ ffmpeg -loglevel debug -fflags nobuffer -rtsp_transport tcp -i 'rtsp://admin:testtest@192.168.1.49:554/h264Preview_01_main' -c copy -f null -
ffmpeg version 4.1.6-1~deb10u1+rpt1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --libdir=/usr/lib/arm-linux-gnueabihf --cpu=arm1176jzf-s --arch=arm
WARNING: library configuration mismatch
avutil configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
avcodec configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
avformat configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
avdevice configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
avfilter configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
avresample configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
swscale configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
swresample configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
postproc configuration: --prefix=/usr --extra-version='1~deb10u1+rpt1' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp --cpu=cortex-a7 --arch=armv6t2 --disable-thumb --enable-shared --disable-doc --disable-programs
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-fflags' ... matched as AVOption 'fflags' with argument 'nobuffer'.
Reading option '-rtsp_transport' ... matched as AVOption 'rtsp_transport' with argument 'tcp'.
Reading option '-i' ... matched as input url with argument 'rtsp://admin:testtest@192.168.1.49:554/h264Preview_01_main'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'null'.
Reading option '-' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url rtsp://admin:testtest@192.168.1.49:554/h264Preview_01_main.
Successfully parsed a group of options.
Opening an input file: rtsp://admin:testtest@192.168.1.49:554/h264Preview_01_main.
[tcp @ 0x13331d0] No default whitelist set
[tcp @ 0x13331d0] Original list of addresses:
[tcp @ 0x13331d0] Address 192.168.1.49 port 554
[tcp @ 0x13331d0] Interleaved list of addresses:
[tcp @ 0x13331d0] Address 192.168.1.49 port 554
[tcp @ 0x13331d0] Starting connection attempt to 192.168.1.49 port 554
[tcp @ 0x13331d0] Successfully connected to 192.168.1.49 port 554
[rtsp @ 0x1331230] SDP:
v=0
o=- 1615111245371232 1 IN IP4 192.168.1.49
s=Session streamed by "preview"
i=h264Preview_01_main
t=0 0
a=tool:LIVE555 Streaming Media v2013.04.08
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Session streamed by "preview"
a=x-qt-text-inf:h264Preview_01_main
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:500
a=rtpmap:96 H264/90000
a=range:npt=0-
a=fmtp:96 packetization-mode=1;profile-level-id=640033;sprop-parameter-sets=Z2QAM6zoAoALWQ==,aO48MA==
a=control:trackID=1
m=audio 0 RTP/AVP 97
c=IN IP4 0.0.0.0
b=AS:256
a=rtpmap:97 MPEG4-GENERIC/16000
a=fmtp:97 streamtype=5;profile-level-id=15;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1408; profile=1;
a=control:trackID=2
Failed to parse interval end specification ''
[rtsp @ 0x1331230] video codec set to: h264
Failed to parse interval end specification ''
[rtsp @ 0x1331230] RTP Packetization Mode: 1
[rtsp @ 0x1331230] RTP Profile IDC: 64 Profile IOP: 0 Level: 33
[rtsp @ 0x1331230] Extradata set to 0x1334b58 (size: 22)
[rtsp @ 0x1331230] audio codec set to: aac
[rtsp @ 0x1331230] audio samplerate set to: 16000
[rtsp @ 0x1331230] audio channels set to: 1
[rtsp @ 0x1331230] setting jitter buffer size to 0
Last message repeated 1 times
[rtsp @ 0x1331230] hello state=0
Failed to parse interval end specification ''
[h264 @ 0x13346e0] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x13346e0] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x13346e0] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x13346e0] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x13346e0] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 0x13346e0] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 0x13346e0] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 0x13346e0] Format yuv420p chosen by get_format().
[h264 @ 0x13346e0] Reinit context to 2560x1440, pix_fmt: yuv420p
[h264 @ 0x13346e0] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 2
Last message repeated 5 times
[rtsp @ 0x1331230] Non-increasing DTS in stream 1: packet 17 with DTS 17408, packet 18 with DTS 16679
[rtsp @ 0x1331230] All info found
[rtsp @ 0x1331230] rfps: 11.750000 0.015265
[rtsp @ 0x1331230] rfps: 11.833333 0.006981
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 11.916667 0.001900
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 12.000000 0.000023
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 12.083333 0.001348
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 12.166667 0.005877
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 12.250000 0.013609
[rtsp @ 0x1331230] rfps: 23.750000 0.016162
[rtsp @ 0x1331230] rfps: 23.833333 0.007602
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 23.916667 0.002245
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 24.000000 0.000091
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 24.083333 0.001141
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 24.166667 0.005393
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 24.250000 0.012849
[rtsp @ 0x1331230] rfps: 36.000000 0.000206
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 48.000000 0.000365
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 60.000000 0.000571
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 120.000000 0.002284
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 240.000000 0.009136
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 23.976024 0.000383
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 59.940060 0.002392
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 11.988012 0.000096
Last message repeated 1 times
[rtsp @ 0x1331230] rfps: 47.952048 0.001531
Last message repeated 1 times
Input #0, rtsp, from 'rtsp://admin:testtest@192.168.1.49:554/h264Preview_01_main':
Metadata:
title : Session streamed by "preview"
comment : h264Preview_01_main
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0, 28, 1/90000: Video: h264 (High), 1 reference frame, yuv420p(progressive, left), 2560x1440, 0/1, 12 tbr, 90k tbn, 180k tbc
Stream #0:1, 39, 1/16000: Audio: aac (LC), 16000 Hz, mono, fltp
Successfully opened the file.
Parsing a group of options: output url -.
Applying option c (codec name) with argument copy.
Applying option f (force format) with argument null.
Successfully parsed a group of options.
Opening an output file: -.
Successfully opened the file.
Output #0, null, to 'pipe:':
Metadata:
title : Session streamed by "preview"
comment : h264Preview_01_main
encoder : Lavf58.20.100
Stream #0:0, 0, 1/90000: Video: h264 (High), 1 reference frame, yuv420p(progressive, left), 2560x1440 (0x0), 0/1, q=2-31, 12 tbr, 90k tbn, 90k tbc
Stream #0:1, 0, 1/16000: Audio: aac (LC), 16000 Hz, mono, fltp
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 17 times
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 13 times
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 13 times
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 4 times
[rtsp @ 0x1331230] CSeq 19 expected, 0 received.4.25 bitrate=N/A speed=1.01x
Last message repeated 1 times
[rtsp @ 0x1331230] CSeq 19 expected, 0 received.4.83 bitrate=N/A speed= 1x
Last message repeated 4 times
[rtsp @ 0x1331230] CSeq 44 expected, 0 received.9.97 bitrate=N/A speed= 1x
Last message repeated 9 times
[rtsp @ 0x1331230] CSeq 44 expected, 0 received.1.69 bitrate=N/A speed= 1x
Last message repeated 4 times
[rtsp @ 0x1331230] CSeq 59 expected, 0 received.6.01 bitrate=N/A speed= 1x
[rtsp @ 0x1331230] CSeq 59 expected, 0 received.6.78 bitrate=N/A speed= 1x
Last message repeated 4 times
[rtsp @ 0x1331230] CSeq 59 expected, 0 received.8.89 bitrate=N/A speed= 1x
Last message repeated 11 times
[rtsp @ 0x1331230] CSeq 84 expected, 0 received.4.54 bitrate=N/A speed= 1x
[rtsp @ 0x1331230] CSeq 99 expected, 0 received.2.11 bitrate=N/A speed= 1x
Last message repeated 5 times
[rtsp @ 0x1331230] CSeq 156 expected, 0 received..87 bitrate=N/A speed= 1x
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87171947
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87172971
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87173995
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87175019
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490344011
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87176043
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490351481
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490358951
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490366511
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87177067
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87178091
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490373981
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87179115
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87180139
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490381451
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490389011
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87181163
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87182187
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490396481
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490403951
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87183211
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87184235
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490411511
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87185259
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87186283
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490418981
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87187307
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 490433759 >= 490426451
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 87188775 >= 87188331
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202388695
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138437240
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202389719
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138444800
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202390743
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202391767
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138452270
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202392791
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138459740
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202393815
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202394839
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138467300
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202395863
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138474770
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202396887
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202397911
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138482240
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202398935
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138489710
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138497270
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202399959
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202400983
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138504740
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202402007
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138512210
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 202423971 >= 202403031
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1138635316 >= 1138519770
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317636742
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317637766
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317638790
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317639814
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786710615
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317640838
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786718085
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317641862
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786725645
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786733115
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317642886
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786740585
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317643910
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317644934
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786748145
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786755615
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317645958
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317646982
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317648006
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786763085
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786770555
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 317657815 >= 317649030
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 1786825419 >= 1786778115
[rtsp @ 0x1331230] CSeq 773 expected, 0 received.9.67 bitrate=N/A speed= 1x
Last message repeated 3 times
[rtsp @ 0x1331230] CSeq 857 expected, 0 received.1.76 bitrate=N/A speed= 1x
Last message repeated 2 times
[rtsp @ 0x1331230] CSeq 895 expected, 0 received.6.78 bitrate=N/A speed= 1x
Last message repeated 2 times
[rtsp @ 0x1331230] CSeq 895 expected, 0 received.7.55 bitrate=N/A speed= 1x
Last message repeated 4 times
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 432892991 >= 432868026
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2435021087 >= 2434883816
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 432892991 >= 432869050
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 432892991 >= 432870074
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2435021087 >= 2434891286
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 432892991 >= 432871098
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 432892991 >= 432872122
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2435021087 >= 2434898846
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 432892991 >= 432873146
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2435021087 >= 2434906316
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 432892991 >= 432874170
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2435021087 >= 2434913786
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 432892991 >= 432875194
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2435021087 >= 2434921346
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2435021087 >= 2434928816
[rtsp @ 0x1331230] CSeq 917 expected, 0 received.2.71 bitrate=N/A speed= 1x
[rtsp @ 0x1331230] CSeq 1000 expected, 0 received..41 bitrate=N/A speed= 1x
Last message repeated 6 times
[rtsp @ 0x1331230] CSeq 1015 expected, 0 received..33 bitrate=N/A speed= 1x
Last message repeated 10 times
[rtsp @ 0x1331230] CSeq 1015 expected, 0 received..34 bitrate=N/A speed= 1x
Last message repeated 14 times
[rtsp @ 0x1331230] CSeq 1024 expected, 0 received..67 bitrate=N/A speed= 1x
[rtsp @ 0x1331230] CSeq 1068 expected, 0 received..72 bitrate=N/A speed= 1x
Last message repeated 2 times
[rtsp @ 0x1331230] CSeq 1081 expected, 0 received..18 bitrate=N/A speed= 1x
Last message repeated 3 times
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 548087929 >= 548051960
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 548087929 >= 548052984
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 548087929 >= 548084754
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 548087929 >= 548085778
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 3082991914 >= 3082800412
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 548087929 >= 548086802
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 1: 548087929 >= 548087826
[null @ 0x13823d0] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 3082991914 >= 3082985670
[rtsp @ 0x1331230] CSeq 1144 expected, 0 received..40 bitrate=N/A speed= 1x
[rtsp @ 0x1331230] CSeq 1144 expected, 0 received..62 bitrate=N/A speed= 1x
Last message repeated 6 times
[rtsp @ 0x1331230] CSeq 1144 expected, 0 received..92 bitrate=N/A speed= 1x
Last message repeated 1 times
[rtsp @ 0x1331230] CSeq 1144 expected, 0 received..52 bitrate=N/A speed= 1x
Last message repeated 7 times
[rtsp @ 0x1331230] CSeq 1158 expected, 0 received..37 bitrate=N/A speed= 1x
Last message repeated 3 times
[rtsp @ 0x1331230] CSeq 1194 expected, 0 received..78 bitrate=N/A speed= 1x
Last message repeated 3 times
[rtsp @ 0x1331230] CSeq 1194 expected, 0 received..89 bitrate=N/A speed= 1x
frame=447480 fps= 12 q=-1.0 Lsize=N/A time=10:04:10.61 bitrate=N/A speed= 1x
video:35977814kB audio:137645kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (rtsp://admin:testtest@192.168.1.49:554/h264Preview_01_main):
Input stream #0:0 (video): 447500 packets read (36842824068 bytes);
Input stream #0:1 (audio): 566060 packets read (140948879 bytes);
Total: 1013560 packets (36983772947 bytes) demuxed
Output file #0 (pipe:):
Output stream #0:0 (video): 447480 packets muxed (36841282003 bytes);
Output stream #0:1 (audio): 566060 packets muxed (140948879 bytes);
Total: 1013540 packets (36982230882 bytes) muxed
0 frames successfully decoded, 0 decoding errors
Exiting normally, received signal 2.
jlpoole@raspberrypi:~
Okay, those Application provided invalid, non monotonically increasing dts
errors look like the same problems that caused Moonfire NVR to close the stream.
Do they happen on the Gentoo server? Do they happen if you remove the -fflags nobuffer
?
The library warning seems to be saying that it compiled the binary and the libraries with this difference in options:
$ diff -u binary avutil
--- binary 2021-03-12 08:57:21.179545106 -0800
+++ avutil 2021-03-12 08:57:21.179363592 -0800
@@ -64,7 +64,10 @@
--enable-chromaprint
--enable-frei0r
--enable-libx264
+--libdir=/usr/lib/arm-linux-gnueabihf/neon/vfp
+--cpu=cortex-a7
+--arch=armv6t2
+--disable-thumb
--enable-shared
---libdir=/usr/lib/arm-linux-gnueabihf
---cpu=arm1176jzf-s
---arch=arm
+--disable-doc
+--disable-programs
I don't think it's causing any problems.
So the "frame=..." is a bottom line in a console streen that is continually updated to show progress. I'm assuming this is an ncurses kind of feature to avoid scrolling caused by repetitive output.
So in my comment earlier above, it looks like something happened within the console (or ncurses?) that caused the update line to move down the screen rather than replace itself so the console looks like it has a constantly updated bottom line. What I thought was an error with ffmpeg looks to have been something caused in my console where the refresh line no longer replaced, but moved down a line in the screen buffer causing the screen to scroll with each line of output.
I now have three consoles running against the same camera, each with a different version of ffmpeg. I have on my taurus laptop the latest & greatest version of ffmpeg, release 4.3.2. I am doing this to see if there are consistent errors in all three session give a certain point of time. So far, I'm seeing different results. I'm now thinking network issues may be in play. The garage cameras come from the garage through a Ubiquiti wireless link, the Peck comes via a long Cat6 running to the house next door, and the laptop has my current house's wireless. So, a variety of network connections.
I eventually found all screens populated with red error messages and then hung. No more updates. I entered Control-C and the ffmpeg program terminated and I was returned to a standard prompt. This proves that my ssh session was still alive and the hang up occurring on ffmpeg's side.
I'm starting to think that moonfire-nvr is faced with handling problems from ffmpeg and being able to recover gracefully when ffmpeg hangs as it appears to have done in several versions on several machines. I'm going to do further testing just using various ffmpegs rtsp sessions, my approach has been haphazard and not very methodical and at this point we need to have some methodically produced results.
The frame=
line is continuously updated just by sending a carriage return, sending the cursor back to the start of the line. If more lines are printed in the meantime(like that CSeq
log line), the display gets a little off as in your screenshot.
There's another option Moonfire NVR sets when it opens streams with ffmpeg, stimeout 10000000
(socket timeout). If a RTSP connection hits that timeout (in microseconds, so 10,000,000 µs = 10 seconds), the connection's dropped, and then Moonfire NVR retries after a second. You can match that with this variation of the command:
ffmpeg -report -loglevel debug -fflags nobuffer -stimeout 10000000 -rtsp_transport tcp -allowed_media_types video -i 'rtsp://...' -c copy -f null -
If your network connection is unreliable, that might just be the best that can be done, with ffmpeg or any other RTSP client. Maybe what changed since you had a working setup on the Pi wasn't ffmpeg or Moonfire but the wireless network conditions.
I've been playing a bit with a Reolink camera (a RLC-410, hardware version IPC_3816M, firmware v2.0.0.1441_19032101). I think it's actually sending interleaved RTP data over a RTSP connection which must be intended for a different RTSP connection:
PLAY
response RTP-Info: ...ssrc=c517011f...
, it started receiving interleaved RTP packets on channel 0 with SSRC=0xc517011f
(as expected) and SSRC=0x5bF73F70
(wtf?). They both have packets coming in with independently incrementing sequence numbers: i.e, one was at 25580, 25581, ... while the other one was at 40477, 40478, ...PLAY
, which is illegal. Even worse, it was before authentication, so it was leaking private data over an unauthenticated session.I imagine these problems are worse if you have multiple NVRs running, record both the main
and sub
streams, and/or have frequent connection drops for other reasons (eg poor wifi).
Once I move to a non-ffmpeg RTSP implementation, I might be able to work around this problem (by discarding data which doesn't have the expected ssrc
) but wow, that's a bug that just shouldn't happen. I wonder if newer models are better (this one's apparently end-of-lifed) and if there's any way to get a bug report to an engineer who would actually fix it.
I also get warnings in the logs about long times to get the next frame if there's a dramatic change to the image. I have the thing in a cabinet with a door right now; if I open or close the door, I can see it in my logs every time. That really shouldn't happen either.
Finally I see cases where the RTP timestamp goes slightly backwards; Moonfire NVR drops the connection in this case. (iirc this also happens with my Hikvision and/or Dahua cameras sometimes.) This seems to happen when they do a time step with SNTP—it must be calculating offsets with CLOCK_REALTIME
rather than CLOCK_MONOTONIC
or some such. If the system clock goes backwards, so do the RTP timestamps. Notably, I currently have the Reolink camera to do a SNTP step every 60 minutes, and the timestamp problems happen almost exactly every two hours. I guess it goes forward half the time and back half the time. I also see a huge mismatch when I first set the clock, so I'm pretty sure this is the problem. I could probably work around this by say assuming an increment of the frame rate if the timestamp delta is unreasonable.
It may not be all the camera's fault, though. The bad cseq
message from ffmpeg is bogus in at least some cases (#126), and we can add support for RTP H.264 NAL unit type 29.
I think I've found something that might bear on the problems I encountered. One of the facts that were different after I started having problems was that I had upgraded my RaspberryPi4 system. This bug gets into the nitty gritty of kernels and sleep modes for USB attached hard drives, and some of the people posting noted their problem went away when the downgraded to a previous kernel. I just wanted to add this URL because it may have bearing on what I experienced: https://github.com/raspberrypi/linux/issues/3981 No need for comments at this time. I'm also grappling with the issue of whether I should replace my SD HC to remove any question about media problems contributing to some dropped videos. I've not been keeping diligent records, but what I am noticing is that my Windows Reolink client is capturing events, triggered by motion settings, at times when moonfire-nvr is not capturing. I basically can use the Reolink software as a control: if it saves files and I cannot find them through moonfire-nvr, then I know something has happened on the RapsberryPi end. I believe my problems are hardware and/or system related and not related to moonfire-nvr.
I am kind of regretting having gone with RaspberryPi since it depends on an SD HC which has an admitted limited life. Even Scott's suggestions tries to limit the cycles performed against the diskette. I noticed some of the of the systems Scott recommended are out-of-stock at the linked vendor. I'm thinking for 24/7 video, something not operating on an SD HC is preferable. Just too may write cycles. And the kernel issue in the bug of the URL herein is troubling. I've also complicated things by using LVM. I used to use LVM for my Xen hypervisor and then I discovered some shortcoming and well as LVM developer malaise that made me decide to go back to the tried-and-true partition approach.
What tangled webs we weave. But having video surveillance feeds coming to a generic browser is so worth it, I just wish I had more time.
I am kind of regretting having gone with RaspberryPi since it depends on an SD HC which has an admitted limited life.
From a previous email, I recall you have a 5-bay USB SATA enclosure. Have you considered buying a SATA SSD and sticking in there? The Pi4 supports USB booting, and a real SATA SSD would be much more reliable.
Good suggestion, but I'm shy about SSDs. Any reason not to use a Western Digital Blue, at $20, it seems like a no-brainer? I don't need the performance an SSD offers; the stability is more important to me. I have two empty slots in my USB array. Thank you.
Edit: changed the URL to points to 3.5" form, original link was to notebook which is a form factor I'm not sure is supported by my array.
I don't think you need to feel shy about SSDs; they're in a different reliability class than SD cards, and they actually report SMART statistics so you can see how much wear they have on them. I'd expect them to be more reliable as a root volume than a refurbished ("renewed") HDD like that one.
But to answer your question: putting the root drive and SQLite database on a HDD is probably fine. Database accesses will be noticeably slower but probably bad enough so that Moonfire NVR will actually break.
Edit: changed the URL to points to 3.5" form, original link was to notebook which is a form factor I'm not sure is supported by my array.
I still see the 2.5" link. Another thing to check is if the 3.5" drive (or whatever you select) is using SMR; I try to avoid those drives.
Usually arrays have trays that let you put 2.5" drives in 3.5" slots. SSDs all come in 2.5" form so you'd need this to take my suggestion of putting in a SSD.
I did more reading about SSD and decided Scott's suggestion was well taken. So I ordered the following:
Installed new Raspberry Pi minimal operating system on PNY. Reconfigured my Pi to boot from USB first (now SD HC is removed). I will be installing (have to buy a CR 1222 battery locally):
The goal here is to isolate potential failures to more reliable systems than an SD HC card that has been in use for over 1 year. Yesterday, on the SD HC-based system, something happened and I could not even reboot from a console, the system locked up. /var/log/messages had some ext4 related errors which causes me to think there were disk problems.
Aug 17 13:07:59 raspberrypi kernel: [55663.494808] Exception stack(0xc923dfa8 to 0xc923dff0)
Aug 17 13:07:59 raspberrypi kernel: [55663.494828] dfa0: 00000020 ad2f50b0 00000020 00000002 00000000 00000000
Aug 17 13:07:59 raspberrypi kernel: [55663.494847] dfc0: 00000020 ad2f50b0 00000000 00000076 ad2f4598 ad2f4574 00000000 ad2f4568
Aug 17 13:07:59 raspberrypi kernel: [55663.494865] dfe0: 00000076 ad2f43a0 b59e1805 b59e34d6
Aug 17 13:07:59 raspberrypi kernel: [55663.494931] task:s-GarageEast-ma state:D stack: 0 pid: 1556 ppid: 1500 flags:0x00000000
Aug 17 13:07:59 raspberrypi kernel: [55663.494962] Backtrace:
Aug 17 13:07:59 raspberrypi kernel: [55663.494996] [<c0b8c820>] (__schedule) from [<c0b8d1f8>] (schedule+0x68/0xe4)
Aug 17 13:07:59 raspberrypi kernel: [55663.495016] r10:00000001 r9:00000000 r8:00000000 r7:005e4859 r6:c33cc050 r5:c2ce3e00
Aug 17 13:07:59 raspberrypi kernel: [55663.495032] r4:ffffe000
Aug 17 13:07:59 raspberrypi kernel: [55663.495053] [<c0b8d190>] (schedule) from [<c057eb60>] (wait_transaction_locked+0xc0/0xf8)
Aug 17 13:07:59 raspberrypi kernel: [55663.495070] r5:c1205048 r4:c33cc000
Aug 17 13:07:59 raspberrypi kernel: [55663.495092] [<c057eaa0>] (wait_transaction_locked) from [<c057f04c>] (start_this_handle+0x4b4/0x970)
Aug 17 13:07:59 raspberrypi kernel: [55663.495110] r7:c3a08360 r6:c0e51834 r5:c33cc028 r4:c33cc000
Aug 17 13:07:59 raspberrypi kernel: [55663.495133] [<c057eb98>] (start_this_handle) from [<c057f610>] (jbd2__journal_start+0x108/0x1fc)
Aug 17 13:07:59 raspberrypi kernel: [55663.495153] r10:00000001 r9:c13fb098 r8:00000000 r7:00000008 r6:00000002 r5:c33cc000
Aug 17 13:07:59 raspberrypi kernel: [55663.495168] r4:c3a08360
Aug 17 13:07:59 raspberrypi kernel: [55663.495194] [<c057f508>] (jbd2__journal_start) from [<c050d568>] (__ext4_journal_start_sb+0x84/0x100)
Aug 17 13:07:59 raspberrypi kernel: [55663.495214] r10:c0c15df4 r9:c0529bd0 r8:c3ae24b0 r7:00000bb6 r6:00000002 r5:00000001
Aug 17 13:07:59 raspberrypi kernel: [55663.495229] r4:c5958400
Aug 17 13:07:59 raspberrypi kernel: [55663.495255] [<c050d4e4>] (__ext4_journal_start_sb) from [<c0532614>] (ext4_da_write_begin+0x1cc/0x498)
Aug 17 13:07:59 raspberrypi kernel: [55663.495273] r7:00000000 r6:01f40000 r5:c3ae23a0 r4:d99538f0
Aug 17 13:07:59 raspberrypi kernel: [55663.495297] [<c0532448>] (ext4_da_write_begin) from [<c03af0c8>] (generic_perform_write+0x108/0x1dc)
Aug 17 13:07:59 raspberrypi kernel: [55663.495317] r10:c0532448 r9:c0c15c24 r8:00000dcd r7:00000233 r6:c3ae24b0 r5:c90adee0
Aug 17 13:07:59 raspberrypi kernel: [55663.495333] r4:00000233
Aug 17 13:07:59 raspberrypi kernel: [55663.495355] [<c03aefc0>] (generic_perform_write) from [<c051b1f8>] (ext4_buffered_write_iter+0xb4/0x184)
Aug 17 13:07:59 raspberrypi kernel: [55663.495375] r10:c90adee0 r9:c1205048 r8:c90adee0 r7:c3ae2428 r6:c3ae23a0 r5:c2ce3e00
Aug 17 13:07:59 raspberrypi kernel: [55663.495390] r4:c90adef8
Aug 17 13:07:59 raspberrypi kernel: [55663.495412] [<c051b144>] (ext4_buffered_write_iter) from [<c051b7a0>] (ext4_file_write_iter+0x120/0x988)
Aug 17 13:07:59 raspberrypi kernel: [55663.495431] r8:c90adf58 r7:00000000 r6:c90adef8 r5:c3ae23a0 r4:00000000
Aug 17 13:07:59 raspberrypi kernel: [55663.495453] [<c051b680>] (ext4_file_write_iter) from [<c043dd64>] (vfs_write+0x234/0x464)
Aug 17 13:07:59 raspberrypi kernel: [55663.495473] r10:00000000 r9:00000000 r8:c90adf58 r7:00000000 r6:c1205048 r5:00033ec2
Aug 17 13:07:59 raspberrypi kernel: [55663.495488] r4:cc96e9c0
Aug 17 13:07:59 raspberrypi kernel: [55663.495509] [<c043db30>] (vfs_write) from [<c043e0f8>] (ksys_write+0x68/0xe0)
Aug 17 13:07:59 raspberrypi kernel: [55663.495529] r10:00033ec2 r9:aafaade8 r8:cc96e9c0 r7:00000000 r6:01f3fdcd r5:cc96e9c3
Aug 17 13:07:59 raspberrypi kernel: [55663.495544] r4:c1205048
Aug 17 13:07:59 raspberrypi kernel: [55663.495564] [<c043e090>] (ksys_write) from [<c043e188>] (sys_write+0x18/0x1c)
Aug 17 13:07:59 raspberrypi kernel: [55663.495583] r10:00000004 r9:c90ac000 r8:c0200204 r7:00000004 r6:accf20b0 r5:aafaade8
Aug 17 13:07:59 raspberrypi kernel: [55663.495599] r4:00033ec2
Aug 17 13:07:59 raspberrypi kernel: [55663.495620] [<c043e170>] (sys_write) from [<c0200040>] (ret_fast_syscall+0x0/0x28)
Aug 17 13:07:59 raspberrypi kernel: [55663.495636] Exception stack(0xc90adfa8 to 0xc90adff0)
Aug 17 13:07:59 raspberrypi kernel: [55663.495655] dfa0: 00033ec2 aafaade8 0000002a aafaade8 00033ec2 00000000
Aug 17 13:07:59 raspberrypi kernel: [55663.495675] dfc0: 00033ec2 aafaade8 accf20b0 00000004 00ba77d0 00033ec2 accf0bd8 aafaade8
Aug 17 13:07:59 raspberrypi kernel: [55663.495693] dfe0: 00000004 accefa50 b59e13ad b59e34d6
Aug 17 13:07:59 raspberrypi kernel: [55663.495757] task:s-PeckAlleyWest state:D stack: 0 pid: 1557 ppid: 1500 flags:0x00000000
As this is now understood as an RTSP-level interop problem with these buggy cameras, let's continue discussion at scottlamb/retina#17 as necessary. Current best solution is to use the new --rtsp-transport=udp argument.
One of four cameras seems to go into a repeated loop stopping and being unable to accept streams and record same. See screenshot below of the web UI where garage_west has an abnormal number of small files
Running moonfire-nvr updated 3/10/2021. Have run previous version, including circa 6/1/2020 where I had several months of operation without this kind of problem.
Desired result: equal recording of camera "garage_west" as three other cameras and web interface offering files of similar size as other cameras
moonfire-nvr@raspberrypi:/usr/local/src/moonfire-nvr/server/target/release $ git describe v0.6.0-40-ge66a88a moonfire-nvr@raspberrypi:/usr/local/src/moonfire-nvr/server/target/release $ ./moonfire-nvr --version moonfire-nvr 0.6.1 moonfire-nvr@raspberrypi:/usr/local/src/moonfire-nvr/server/target/release $
Ran with RUST_BACKTRACE=1
log ( 254MB; feel free to select snippets and post herein as I'll remove this large file after 1 week) for approximately 4 hour session this afternoon at:
https://drive.google.com/file/d/1Ir0SZ3jlsiLO5IiyMELQnX4YkJp_Akxk/view?usp=sharing
Screenshot showing the Web interface and the garage_west files constantly restarting![Screenshot_2021-03-10_1728_ Moonfire_NVR](https://user-images.githubusercontent.com/227313/110723082-57b73980-81c8-11eb-9910-1aa1286e5323.png)
As a check, I have the Reolink client software running and capturing events from garage_west and the three other cameras and there has been no noticeable difference for garage_west's camera. I therefore think something on the receiving side, e.g. moonfire-nvr on my Raspberry Pi4 is causing the problem.