search

thirtythreeforty / neolink

An RTSP bridge to Reolink IP cameras
https://www.thirtythreeforty.net/posts/2020/05/hacking-reolink-cameras-for-fun-and-profit/
GNU Affero General Public License v3.0
890 stars 149 forks source link

Can't stream h265 from D800 #118

Open richardklingler opened 3 years ago

richardklingler commented 3 years ago

Describe the bug Neolink shows loads of warning messages when connecting to a D800 camera, therefore unable to stream h265 mainStream. h264 subStream works fine.

To Reproduce Steps to reproduce the behavior. Example:

  1. Add D800 camera
  2. Launch Neolink with debugging option

Neolink log:

0:00:04.484595215 17066 0x7f5ea0005c00 WARN default gstrtspconnection.c:3879:gst_rtsp_watch_write_data: too much backlog: max_bytes 0, current 140400, max_messages 100, current 100 0:00:04.484659904 17066 0x7f5ea0005c00 WARN default gstrtspconnection.c:3879:gst_rtsp_watch_write_data: too much backlog: max_bytes 0, current 140400, max_messages 100, current 100 0:00:04.484733234 17066 0x7f5ea0005c00 WARN default gstrtspconnection.c:3879:gst_rtsp_watch_write_data: too much backlog: max_bytes 0, current 140400, max_messages 100, current 100 0:00:04.484752604 17066 0x7f5ea0005c00 WARN default gstrtspconnection.c:3879:gst_rtsp_watch_write_data: too much backlog: max_bytes 0, current 140400, max_messages 100, current 100 0:00:04.485429498 17066 0x7f5ea0005c00 FIXME h265parse gsth265parse.c:1792:gst_h265_parse_parse_frame: Implement timestamp/duration interpolation based on SEI message 0:00:04.558547560 17066 0x7f5ea0005c00 FIXME h265parse gsth265parse.c:1792:gst_h265_parse_parse_frame: Implement timestamp/duration interpolation based on SEI message 0:00:04.558953956 17066 0x7f5ea0005c00 FIXME h265parse gsth265parse.c:1792:gst_h265_parse_parse_frame: Implement timestamp/duration interpolation based on SEI message

VLC log:

[00007f445008aea0] hevc packetizer: Waiting for VPS/SPS/PPS [00007f445008aea0] hevc packetizer: Waiting for VPS/SPS/PPS [00007f445008aea0] hevc packetizer: Waiting for VPS/SPS/PPS [00007f445008aea0] hevc packetizer: Waiting for VPS/SPS/PPS

Expected behavior Unable to play rtsp stream via vlc or record through shinobi.

Versions NVR software: VLC, Shinobi Neolink software: 0.3.0 Reolink camera model and firmware: D800, v3.0.0.82_20080601

QuantumEntangledAndy commented 3 years ago

This one is saying that we are recieving data (from the camera) faster that that data is being pulled (from the watching client) and that the queue has filled up. Is suspect that it's a bandwidth issue. What is the resolution and the frame rate of the mainStream? That may be too much data to push over your network.

Unlike the official client neolink tries to send all of the frames without dropping any. This is perhaps something we could change but would need to do it carefully to avoid visual artifacts in the stream.

In the mean time you could try reducing the frame rate or resolution of the HD stream.

QuantumEntangledAndy commented 3 years ago

I only have an E1 so I don't have this issue and thirtythreeforty has a D400 and a Lumus I think. Although we try to support all cameras we can only really say it works with things we have tested it on.

If a developer had a D800 camera they could probably do some more involved testing for you and see how much data we get from a single frame and be more certain of the cause.

If your good with programming please consider joining in and helping us to develop for the D800.

QuantumEntangledAndy commented 3 years ago

Also VLC has quite a small default buffer size. Too small for HD camera streams. You will need to increase it to connect using VLC (or ensure you connect over TCP rather than the default UDP).

twistedddx commented 3 years ago

Ithirtythreeforty has b800's so Neolink has been tested to work with them and originally deigned to work with them.

QuantumEntangledAndy commented 3 years ago

I see well I think my other comments still stand. Ensure you have enough bandwidth and buffer. Perhaps thirtythreeforty could comment on his setup that is working.

richardklingler commented 3 years ago

Okay...sorry for late reply....time zone differences (o;

My setup:

Intel NUC10i7 running latest shinobi pro under Ubuntu Server 0.04.1 On the same machine neolink 0.3.0 connecting to D800 on local network (all 1GB connections).

When neolink is running I can connect to it from another workstation with vlc watching both main and sub stream at the same time without problems...

Now I add in shinobi a new monitor with main stream for recording and sub stream for local viewing:

main stream: rtsp://127.0.0.1:8554/office
sub stream:  rtsp://127.0.0.1:8554/office/subStream

Live view works for half a minute, then stops...also recordings seem to be in very slow motion as the timestamp visible in the camera advances every 5 or 10 seconds....and recording files are way smaller than they should be....

Also when connecting from another machine I get with vlc:

me@blender:$ vlc "rtsp://10.0.100.22:8554/office" --zoom=0.25
VLC media player 3.0.11 Vetinari (revision 3.0.11-0-gdc0c5ced72)
[000055bd1c630570] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Created new TCP socket 31 for connection
Created new TCP socket 30 for connection
[00007f50b8001130] live555 demux error: no data received in 10s, aborting
Created new TCP socket 30 for connection
Created new TCP socket 30 for connection
[00007f50b80987b0] live555 demux error: no data received in 10s, aborting
Created new TCP socket 30 for connection

Substream is still watchable remotely....

ffprobe also takes over 12 seconds to respond what stream it is....

As soon I remove the monitor settings inside shinobi...everything is normal again....

richardklingler commented 3 years ago

Also when recording manually with ffmpeg I get video where it stucks from time to time with constant messages below:

me@blender:~$ ffmpeg -i rtsp://10.0.100.22:8554/office -fflags +igndts -copytb 1 -c:v copy -map 0 -f segment -segment_time 300 -segment_format mp4 "Videos/capture-office-%03d.mp4"
ffmpeg version 4.1.6-1~deb10u1 Copyright (c) 2000-2020 the FFmpeg developers
Guessed Channel Layout for Input Stream #0.1 : stereo
Input #0, rtsp, from 'rtsp://10.0.100.22:8554/office':
  Metadata:
    title           : Session streamed with GStreamer
    comment         : rtsp-server
  Duration: N/A, start: 0.064000, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuvj420p(pc, bt709), 3840x2160, 20 fps, 20 tbr, 90k tbn, 20 tbc
    Stream #0:1: Audio: pcm_s16be, 16000 Hz, stereo, s16, 512 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (pcm_s16be (native) -> aac (native))
Press [q] to stop, [?] for help
[segment @ 0x55752964c3c0] Opening 'Videos/capture-office-000.mp4' for writing
Output #0, segment, to 'Videos/capture-office-%03d.mp4':
  Metadata:
    title           : Session streamed with GStreamer
    comment         : rtsp-server
    encoder         : Lavf58.20.100
    Stream #0:0: Video: hevc (Main), yuvj420p(pc, bt709), 3840x2160, q=2-31, 20 fps, 20 tbr, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac (LC), 16000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc58.35.100 aac
[segment @ 0x55752964c3c0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packet
[rtsp @ 0x5575295f59c0] RTP: missed 10 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.16x    
[rtsp @ 0x5575295f59c0] RTP: missed 131 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.14x    
[rtsp @ 0x5575295f59c0] RTP: missed 7 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packet
[rtsp @ 0x5575295f59c0] RTP: missed 8 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.09x    
[rtsp @ 0x5575295f59c0] RTP: missed 244 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.08x    
[rtsp @ 0x5575295f59c0] RTP: missed 1 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.05x    
[rtsp @ 0x5575295f59c0] RTP: missed 3 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.04x    
[rtsp @ 0x5575295f59c0] RTP: missed 2 packets
[rtsp @ 0x5575295f59c0] jitter buffer full0:00:35.86 bitrate=N/A speed=1.03x    
[rtsp @ 0x5575295f59c0] RTP: missed 19 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packet
[rtsp @ 0x5575295f59c0] RTP: missed 88 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.03x    
[rtsp @ 0x5575295f59c0] RTP: missed 9 packets
[aac @ 0x55752964cdc0] Queue input is backward in timeitrate=N/A speed=1.03x    
[segment @ 0x55752964c3c0] Non-monotonous DTS in output stream 0:1; previous: 695020, current: 694326; changing to 695021. This may result in incorrect timestamps in the output file.
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.02x    
[rtsp @ 0x5575295f59c0] RTP: missed 11 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.02x    
[rtsp @ 0x5575295f59c0] RTP: missed 2 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.02x    
[rtsp @ 0x5575295f59c0] RTP: missed 10 packets
[aac @ 0x55752964cdc0] Queue input is backward in timeitrate=N/A speed=1.02x    
[segment @ 0x55752964c3c0] Non-monotonous DTS in output stream 0:1; previous: 995760, current: 994758; changing to 995761. This may result in incorrect timestamps in the output file.
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packet
[rtsp @ 0x5575295f59c0] RTP: missed 4 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.02x    
[rtsp @ 0x5575295f59c0] RTP: missed 8 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.01x    
[rtsp @ 0x5575295f59c0] RTP: missed 164 packets
[aac @ 0x55752964cdc0] Queue input is backward in timeitrate=N/A speed=1.01x    
[segment @ 0x55752964c3c0] Non-monotonous DTS in output stream 0:1; previous: 1353798, current: 1353237; changing to 1353799. This may result in incorrect timestamps in the output file.
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.01x    
[rtsp @ 0x5575295f59c0] RTP: missed 10 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.01x    
[rtsp @ 0x5575295f59c0] RTP: missed 242 packets
[rtsp @ 0x5575295f59c0] max delay reached. need to consume packetspeed=1.01x
QuantumEntangledAndy commented 3 years ago

I know this is late but thought I'd comment for anyone coming here. This log about ffempg and max delay recached is to do with large amount of data coming from the camera. You should be able resolve this by switching ffmpeg to prefer TCP over UDP. Or substantially increasing ffmpeg UDP buffer size. TCP is still a better choice if you want quality recordings though.

r4mtor commented 3 years ago

I have similar problems with the D800 and the pi. neolinks connects to cam but the stream doesnt't work. If someone could help...

ubuntu@pi:~/repos/neolink$ export RUST_LOG=info && ./target/release/neolink --config neo.toml
[2021-02-11T22:55:54Z INFO  neolink] Neolink v0.3.0-164-g5c83472 release
[2021-02-11T22:55:54Z INFO  neolink] reno: Connecting to camera at 192.168.188.56:9000
[2021-02-11T22:55:54Z INFO  neolink] reno: Connected and logged in
[2021-02-11T22:55:54Z INFO  neolink] reno: Camera time is already set: 2021-02-11 23:55:54 +1
[2021-02-11T22:55:54Z INFO  neolink] reno: Camera reports firmware version v3.0.0.183_21012801
[2021-02-11T22:55:54Z INFO  neolink] reno: Starting video stream mainStream
[2021-02-11T22:54:44Z DEBUG neolink::gst] RTSP: media was configured

Try to connect with ffprobe

ubuntu@pi:~/repos/neolink$ ffprobe rtsp://192.168.188.20:8554/reno
ffprobe version 4.2.4-1ubuntu0.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[rtsp @ 0xaaaaebd0fd30] method DESCRIBE failed: 503 Service Unavailable
rtsp://192.168.188.20:8554/reno: Server returned 5XX Server Error reply

And here is me config

bind = "0.0.0.0"

[[cameras]]
name = "reno"
username = "admin"
password = "1234567890"
address = "192.168.188.56:9000"
stream = "mainStream"

Maybe I could help debuging

QuantumEntangledAndy commented 3 years ago

@r4mtor I suspect your issue if different. Please open a new issue.

In the new issue post the results of the gstreamer debug log with this

GST_DEBUG=3 ./neolink --config my_config.toml

Also please read the doc in doc/unix_setup.md it may help you.