blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.57k stars 1.8k forks source link

[Config Support]: Stutter on recordings using Tapo C325WB and PICAM's #8225

Closed alsimms closed 1 year ago

alsimms commented 1 year ago

Describe the problem you are having

First off my hat's off to all the devs, contributors and testers that work and support this incredible project.

My issue is related to stutter in all recordings no matter from which camera it comes from. Playing video from camera directly plays fine. Playing video using live webrtc plays fine with no stutter. I have searched here and found some custom settings but these are mostly for reo cams and when I tried, it complained about invalid options and settings being used.

Everything words perfect as expected, it is just the recording portion that needs help. Thanks in advance.

Frigate is running on an 8400T cpu with 6 cores, 16 GB ram and 512GB m.2 msata 512GB.

Docker compose

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "300mb" # update for your cameras based on calculation above
    devices:
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /mnt/SHNET-POOL1/Docker/frigate/config/config.yml:/config/config.yml
      - /mnt/SHNET-POOL1/Docker/frigate/storage:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp

ffprobe output from camera.

ffprobe rtsp://xxxxxx:xxxxxx@xxx.xxx.xxx.xxx/stream1
ffprobe version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  WARNING: library configuration mismatch
  avcodec     configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --enable-version3 --disable-doc --disable-programs --enable-libaribb24 --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libtesseract --enable-libvo_amrwbenc --enable-libsmbclient
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from 'rtsp://xxxx:xxxxx@xxx.xxx.xxx.xxx/stream1':
  Metadata:
    title           : Session streamed by "TP-LINK RTSP Server"
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuvj420p(pc, progressive), 2688x1520, 20 fps, 20 tbr, 90k tbn, 40 tbc
  Stream #0:1: Audio: pcm_alaw, 8000 Hz, 1 channels, s16, 64 kb/s

Version

0.12.1-367d724

Frigate config file

mqtt:
  enabled: True
  host: xxx.xxx.xxx.xxx
  topic_prefix: frigate  
  user: xxxxx
  password: xxxx 
  stats_interval: 15

detectors:
  ov:
    type: openvino
    device: AUTO
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

ffmpeg:
  global_args: -hide_banner -loglevel warning -threads 2
  hwaccel_args: preset-intel-qsv-h264
  output_args:
    record: preset-record-generic-audio-copy

ui:
  live_mode: webrtc
birdseye:
  enabled: false

record:
  enabled: True
  retain:
    days: 14
    mode: motion
  events:
    pre_capture: 2
    retain:
      default: 30
      mode: active_objects
timestamp_style:
  position: "tr"
  effect: solid
snapshots:
  enabled: True
  clean_copy: True
  timestamp: False
  crop: True
  bounding_box: True
  retain:
    default: 30
detect:
  enabled: True
  fps: 5
  width: 1280
  height: 720
objects:
  track:
    - person
    - car
    - bicycle
    - motorcycle
    - dog
    - cat
    - bus
    - boat
    - bird
    - horse
    - bear

go2rtc:
  streams:
     tapofront:
       - rtsp://xxx.xxx.xxx.xxx/stream1
       - "ffmpeg:tapofront#audio=aac"
     tapofront_sub:
       - rtsp://xxx.xxx.xxx.xxx/stream2
     tapodeck:
       - rtsp://xxx.xxx.xxx.xxx/stream1
       - "ffmpeg:tapodeck#audio=aac"
     tapodeck_sub:
       - rtsp://xxx.xxx.xxx.xxx/stream2
     taposwing:
       - rtsp://xxx.xxx.xxx.xxx/stream1
       - "ffmpeg:taposwing#audio=aac"
     taposwing_sub:
       - rtsp://xxx.xxx.xxx.xxx/stream2
     garage:
       - rtsp://xxx.xxx.xxx.xxx:554/h264
     roofcam:
       - rtsp://xxx.xxx.xxx.xxx:9554/roofcam

  webrtc:
    candidates:
      - xxx.xxx.xxx.xxx:8555
      - stun:8555

cameras:
  tapodeck: # <------ Name the camera
    motion:
      mask:
        - 0,188,494,35,796,34,955,54,1280,139,1280,0,0,0
        - 499,663,581,614,558,440,523,371,457,326,298,331,181,371,192,586,227,630
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/tapodeck_sub # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/tapodeck # <----- The stream you want to use for recording
          roles:
            - record 
    live:
      stream_name: tapodeck_sub

  tapofront: # <------ Name the camera
    motion:
      mask:
        - 1280,0,1280,313,1214,265,1220,216,1259,147,1252,60,1187,38,1078,34,982,61,801,63,760,125,279,131,151,141,0,73,0,0
        - 1106,720,954,496,778,408,412,410,339,529,344,720
    ffmpeg: 
      inputs:
        - path: rtsp://127.0.0.1:8554/tapofront_sub # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/tapofront # <----- The stream you want to use for recording
          roles:
            - record
    live:
      stream_name: tapofront_sub

  taposwing: # <------ Name the camera
    motion:
      mask:
        - 587,40,981,48,1280,74,1280,0,605,0,508,0,358,0,363,41
    ffmpeg:

      inputs:
        - path: rtsp://127.0.0.1:8554/taposwing_sub # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/taposwing # <----- The stream you want to use for recording
          roles:
            - record
    live:
      stream_name: taposwing_sub

  garage: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage # <----- The stream you want to use for detection
          roles:
            - detect
            - record
    live:
      stream_name: garage

  roofcam: # <------ Name the camera
    motion:
      mask:
        - 617,0,482,149,343,187,369,324,809,406,1135,377,1160,0
        - 0,585,0,633,78,634,212,509,276,412,368,307,280,125,193,105,153,102,97,118,76,143,0,158,0,227
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/roofcam # <----- The stream you want to use for detection
          roles:
            - detect
            - record
    live:
      stream_name: roofcam

Relevant log output

There are no errors in the logs.

Frigate stats

cpu_usages  
1   
cpu "0.0"
mem "0.0"
15  
cpu "0.0"
mem "0.0"
16  
cpu "0.0"
mem "0.0"
24  
cpu "0.0"
mem "0.0"
25  
cpu "0.0"
mem "0.0"
26  
cpu "0.0"
mem "0.0"
27  
cpu "0.0"
mem "0.0"
28  
cpu "0.0"
mem "0.0"
29  
cpu "0.0"
mem "0.0"
30  
cpu "0.0"
mem "0.0"
31  
cpu "0.0"
mem "0.0"
32  
cpu "0.0"
mem "0.0"
41  
cpu "0.0"
mem "0.0"
42  
cpu "0.0"
mem "0.0"
80  
cpu "0.0"
mem "0.0"
81  
cpu "0.0"
mem "0.0"
82  
cpu "0.0"
mem "0.0"
89  
cpu "20.0"
mem "0.1"
97  
cpu "0.0"
mem "0.0"
98  
cpu "5.7"
mem "1.3"
119 
cpu "0.0"
mem "0.1"
134 
cpu "0.0"
mem "0.0"
135 
cpu "0.0"
mem "0.0"
136 
cpu "0.0"
mem "0.0"
137 
cpu "0.0"
mem "0.0"
138 
cpu "0.0"
mem "0.0"
143 
cpu "0.0"
mem "0.0"
353 
cpu "0.3"
mem "0.5"
359 
cpu "1.0"
mem "0.1"
360 
cpu "0.0"
mem "2.8"
362 
cpu "2.7"
mem "0.6"
365 
cpu "1.0"
mem "0.7"
366 
cpu "0.0"
mem "0.1"
367 
cpu "1.3"
mem "0.6"
368 
cpu "0.0"
mem "0.1"
369 
cpu "0.7"
mem "0.7"
370 
cpu "1.3"
mem "0.6"
371 
cpu "0.0"
mem "0.1"
372 
cpu "1.0"
mem "0.6"
373 
cpu "3.0"
mem "0.6"
375 
cpu "3.0"
mem "0.6"
377 
cpu "0.0"
mem "0.1"
381 
cpu "3.0"
mem "0.6"
382 
cpu "4.3"
mem "0.3"
384 
cpu "0.0"
mem "0.1"
386 
cpu "3.3"
mem "0.6"
391 
cpu "3.0"
mem "0.6"
402 
cpu "4.0"
mem "0.3"
404 
cpu "1.7"
mem "0.2"
405 
cpu "2.0"
mem "0.2"
406 
cpu "5.3"
mem "0.4"
408 
cpu "4.0"
mem "0.3"
447 
cpu "3.0"
mem "0.2"
510 
cpu "3.7"
mem "0.1"
513 
cpu "4.3"
mem "0.1"
527 
cpu "3.7"
mem "0.1"
1062    
cpu "3.7"
mem "0.3"
9661    
cpu "0.0"
mem "0.0"
9688    
cpu "0.0"
mem "0.0"
9690    
cpu "0.3"
mem "0.0"
9692    
cpu "0.3"
mem "0.0"
9693    
cpu "1.0"
mem "0.0"
%Cpu(s):    
cpu "id,"
mem "1.1"
MiB 
cpu "13614.4"
mem "avail"
PID 
cpu "%CPU"
mem "%MEM"
Tasks:  
cpu "stopped,"
mem "0"
top 
cpu "users,"
mem "load"
detection_fps   0.2
detectors   
ov  
detection_start 0
inference_speed 15.19
pid 360
garage  
camera_fps  5
capture_pid 386
detection_enabled   1
detection_fps   0
ffmpeg_pid  402
pid 370
process_fps 5
skipped_fps 0
gpu_usages  
intel-qsv   
gpu "4.84 %"
mem "- %"
roofcam 
camera_fps  5.1
capture_pid 391
detection_enabled   1
detection_fps   0
ffmpeg_pid  406
pid 372
process_fps 5.1
skipped_fps 0
service 
last_updated    1697662091
latest_version  "0.12.1"
storage 
/dev/shm    
free    305.2
mount_type  "tmpfs"
total   314.6
used    9.3
/media/frigate/clips    
free    4900388.9
mount_type  "nfs4"
total   15685802.1
used    10785413.2
/media/frigate/recordings   
free    4900388.9
mount_type  "nfs4"
total   15685802.1
used    10785413.2
/tmp/cache  
free    984
mount_type  "tmpfs"
total   1000
used    16
temperatures    {}
uptime  5580
version "0.12.1-367d724"
tapodeck    
camera_fps  5.1
capture_pid 373
detection_enabled   1
detection_fps   0
ffmpeg_pid  1062
pid 365
process_fps 5.1
skipped_fps 0
tapofront   
camera_fps  5.1
capture_pid 375
detection_enabled   1
detection_fps   0.2
ffmpeg_pid  382
pid 367
process_fps 5.1
skipped_fps 0
taposwing   
camera_fps  5
capture_pid 381
detection_enabled   1
detection_fps   0
ffmpeg_pid  408
pid 369
process_fps 5
skipped_fps 0

Operating system

Debian

Install method

Docker Compose

Coral version

M.2

Any other information that may be helpful

No response

NickM-27 commented 1 year ago

Please read the docs, you are using the restream but not using preset-rtsp-restream for the input_args. This is one of the primary reasons separate args are recommended - to avoid stuttering in recordings https://docs.frigate.video/configuration/restream#reduce-connections-to-camera

alsimms commented 1 year ago

Hi Nick,

Thank you for your prompt reply. My mistake, I have added the preset-rtsp-restream to input_args to all main streams and substreams but the issue still remains after testing again. It is also my understanding that under the current config, rtc2go is the only connection to the cameras and frigate is pulling from 8554 on local host. There is nothing else connecting to the cameras.

The config portion in question now looks like this.

cameras:
  tapodeck: # <------ Name the camera
    motion:
      mask:
        - 0,188,494,35,796,34,955,54,1280,139,1280,0,0,0
        - 499,663,581,614,558,440,523,371,457,326,298,331,181,371,192,586,227,630
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/tapodeck_sub # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/tapodeck # <----- The stream you want to use for recording
          input_args: preset-rtsp-restream
          roles:
            - record 
    live:
      stream_name: tapodeck_sub

  tapofront: # <------ Name the camera
    motion:
      mask:
        - 1280,0,1280,313,1214,265,1220,216,1259,147,1252,60,1187,38,1078,34,982,61,801,63,760,125,279,131,151,141,0,73,0,0
        - 1106,720,954,496,778,408,412,410,339,529,344,720
    ffmpeg: 
      inputs:
        - path: rtsp://127.0.0.1:8554/tapofront_sub # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/tapofront # <----- The stream you want to use for recording
          input_args: preset-rtsp-restream
          roles:
            - record
    live:
      stream_name: tapofront_sub

  taposwing: # <------ Name the camera
    motion:
      mask:
        - 587,40,981,48,1280,74,1280,0,605,0,508,0,358,0,363,41
    ffmpeg:

      inputs:
        - path: rtsp://127.0.0.1:8554/taposwing_sub # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/taposwing # <----- The stream you want to use for recording
          input_args: preset-rtsp-restream
          roles:
            - record
    live:
      stream_name: taposwing_sub

  garage: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record
    live:
      stream_name: garage

  roofcam: # <------ Name the camera
    motion:
      mask:
        - 617,0,482,149,343,187,369,324,809,406,1135,377,1160,0
        - 0,585,0,633,78,634,212,509,276,412,368,307,280,125,193,105,153,102,97,118,76,143,0,158,0,227
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/roofcam # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record
    live:
      stream_name: roofcam
NickM-27 commented 1 year ago

to be clear, new recordings after making the change and restarting frigate still have the stuttering?

I would suggest using preset-vaapi instead of qsv

alsimms commented 1 year ago

Yes, after restarting they do still stutter. I will try your recommendation and get back you in a few. Thank you once more !

alsimms commented 1 year ago

Sorry Nick,

I made the changes you suggested but after restarting and testing with preset-vaapi the pauses and catch ups became more pronounced.

NickM-27 commented 1 year ago

okay, then I'd suggest trying to put ffmpeg: in front of all the go2rtc streams

alsimms commented 1 year ago

All Right Boss,

You got it, give a few and I will get it done ! :rofl:

alsimms commented 1 year ago

Do I need to add quotes like the one below it? - "ffmpeg:tapofront#audio=aac" or is it fine without them?

I tested as is below and still have the issue, no error on log.

go2rtc:
  streams:
     tapofront:
       - ffmpeg:rtsp://xxx.xxx.xxx.xxx/stream1
       - "ffmpeg:tapofront#audio=aac"
     tapofront_sub:
       - ffmpeg:rtsp://xxx.xxx.xxx.xxx/stream2
     tapodeck:
       - ffmpeg:rtsp://xxx.xxx.xxx.xxx/stream1
       - "ffmpeg:tapodeck#audio=aac"
     tapodeck_sub:
       - ffmpeg:rtsp://xxx.xxx.xxx.xxx/stream2
     taposwing:
       - ffmpeg:rtsp://xxx.xxx.xxx.xxx/stream1
       - "ffmpeg:taposwing#audio=aac"
     taposwing_sub:
       - ffmpeg:rtsp://xxx.xxx.xxx.xxx/stream2
     garage:
       - ffmpeg:rtsp://xxx.xxx.xxx.xxx:554/h264
     roofcam:
       - ffmpeg:rtsp://xxx.xxx.xxx.xxx:9554/roofcam
NickM-27 commented 1 year ago

Yeah I'd suggest adding quotes. If that doesn't work then I don't know what it would be other than camera iframe issues but I don't believe that's configurable on tapos

alsimms commented 1 year ago

Nick,

That did not resolved it either. I think you are correct in your thinking about iframes or -use_wallclock_as_timestamps which has been discussed a lot with Reo cams. This has been my suspicion all along. Most people resolved these same issues by adding...

-f
live_flv

to their configs but I do not think that applies in my case as Reo cams have a link with FLV and mine does not. Regardless, I appreciate your super duper help. The project is lucky to have you ! You would be an asset in any organization. I wish you nothing but the best!

See you later alligator !

NickM-27 commented 1 year ago

Feel free to leave this open, some others may have ideas to help