search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
17.99k stars 1.64k forks source link

[Support]: Event/Recording Playback Stuttering #3270

Closed esmoyer closed 2 years ago

esmoyer commented 2 years ago

Describe the problem you are having

I upgraded my cameras from wireless Eufy's to wired Amcrest in hopes of solving this issue, thinking it was due to wifi; but sadly no luck.

I am having the same issues as listed from the previous issues below. I have tried the recommendations and nothing seems to help. I assume it's because they had Reolinks and I have Amcrests. I have tried playing around with the input arguments but nothing seems to work. Just commented them out for now, though I'm pretty sure they are the defaults anyway.

Viewing from the cam web server or VLC directly yields no stuttering. Would appreciate any additional input on anything I might be missing.

https://github.com/blakeblackshear/frigate/issues/1499 (my videos and playback buffering look exactly like the ones in this issue) https://github.com/blakeblackshear/frigate/issues/896

Version

0.11.0-d2c3cdc

Frigate config file

mqtt:
  host: XXXX
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: mqtt
  password: XXXX

detectors:
  coral:
    type: edgetpu
    device: usb

database:
  path: /media/frigate/db/frigate.db

logger:
  default: info
  # logs:
    # frigate.event: debug

birdseye:
  enabled: true
  width: 1280
  height: 720
  quality: 4  # 1 is the highest quality, and 31 is the lowest.
  mode: motion

ui:
  use_experimental: true

cameras:
########## CAMERAS
#################### DRIVEWAY
  Driveway:
    ffmpeg:
      inputs:
        - path: rtsp://X:X@10.X.X.3
          roles:
            - detect
            - record
      global_args: 
        -hide_banner 
        -loglevel error
      # hwaccel_args: 
        # - -hwaccel
        # - vaapi
        # - -hwaccel_device
        # - /dev/dri/renderD128
        # - -hwaccel_output_format
        # - yuv420p
      # input_args: 
        # -avoid_negative_ts make_zero 
        # -fflags +genpts+discardcorrupt 
        # -rtsp_transport tcp 
        # -stimeout 5000000 
        # -use_wallclock_as_timestamps
        # - 1
      output_args:
        record: 
          -f segment 
          -segment_time 10 
          -segment_format mp4 
          -reset_timestamps 1 
          -strftime 1 
          -c copy 
          #-an

    detect:
      enabled: true 
      max_disappeared: 30
      width: 2592
      height: 1944
      fps: 5
      stationary:
        interval: 0
        threshold: 50
        # max_frames:
          # default: 3000

    objects: 
      track:
        - person
        - car
        - dog
      filters:
        person:
          min_area: 1100
          max_area: 400000
          min_score: 0.68
          threshold: 0.74
          mask:
            - 22,328,107,324,118,292,21,305
        dog:
          min_area: 1000
          max_area: 50000
          min_score: 0.60
          threshold: 0.65
        car:
          min_area: 5000
          max_area: 250000
          min_score: 0.67
          threshold: 0.73
          mask:
            - 77,320,329,265,538,178,531,73,318,127
            - 1228,209,1430,221,1598,226,1542,133,1484,75,1256,41,1211,95

    motion:
      threshold: 25
      improve_contrast: true
      mask:
        - 1654,0,1920,0,2570,0,2426,351,1927,178,1636,84
        - 2592,1944,2592,1540,2368,1550,2112,1570,2148,1944
        - 157,157,228,105,390,0,0,0,0,361

    record:
      enabled: true
      retain:
        days: 7
        mode: all
      events:
        max_seconds: 600
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - dog
          - car
        retain:
          default: 21
          objects:
            car: 7

    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      crop: false 
      retain:
        default: 21

    rtmp:
      enabled: false

    timestamp_style:
      position: "bl"
      thickness: 2

    best_image_timeout: 30

    zones:  
      zone_driveway:
        coordinates: 987,1492,2282,1430,2592,1357,2592,1138,2404,935,2209,753,1778,729,1058,710

      zone_driveway_entry:
        coordinates: 2245,781,2163,699,1064,643,1039,738

      zone_front_lawn:
        coordinates: 1009,609,664,630,243,656,0,707,0,1323,396,1404,910,1477

      zone_front_sidewalk:
        coordinates: 0,647,555,585,970,566,1512,596,2338,686,2247,634,1966,591,1538,544,1064,523,744,525,252,566,0,600

      zone_street:
        coordinates: 277,335,0,415,0,548,449,458,1013,443,1576,471,2295,578,2592,652,2514,531,2150,419,1852,357,1449,297,961,275,578,292

      zone_adamally_driveway_entry:
        coordinates: 71,329,213,292,335,269,363,243,204,273,112,290

      zone_janejodi_driveway_entry:
        coordinates: 1232,178,1224,200,1636,239,1611,213      

################### DOORBELL
  Doorbell:
    ffmpeg:
      inputs:
        - path: rtsp://X:X@10.X>X.7:554/cam/realmonitor?channel=1&subtype=0&authbasic=64
          roles:
            - detect
            - record
      global_args: 
        -hide_banner 
        -loglevel error
      # hwaccel_args: 
        # - -hwaccel
        # - vaapi
        # - -hwaccel_device
        # - /dev/dri/renderD128
        # - -hwaccel_output_format
        # - yuv420p
      # input_args: 
        # -avoid_negative_ts make_zero 
        # -fflags +genpts+discardcorrupt 
        # -rtsp_transport tcp 
        # -stimeout 5000000 
        # -use_wallclock_as_timestamps 1
      output_args:
        record: 
          -f segment 
          -segment_time 10 
          -segment_format mp4 
          -reset_timestamps 1 
          -strftime 1 
          -c copy 
          #-an

    detect:
      enabled: true
      max_disappeared: 30
      width: 2560
      height: 1920
      fps: 5

    objects:
      track:
        - person
        - dog
      filters:
        person:
          min_area: 2000
          max_area: 450000
          min_score: 0.67
          threshold: 0.74
        dog:
          min_area: 2000
          max_area: 100000
          min_score: 0.65
          threshold: 0.70

    motion:
      threshold: 25
      improve_contrast: true
      mask:
        - 830,950,654,1024,325,967,0,1013,0,0,520,0,858,0,1341,0,2560,0,2479,889,2490,1006,1309,914
        - 2281,1920,2443,1278,2302,1381,1786,1503,1657,1638,1664,1833,1700,1920
        - 779,1541,728,1299,487,1231,412,1519

    record:
      enabled: true
      retain:
        days: 7
      events:
        #max_seconds: 300
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - dog
        retain:
          default: 21

    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      crop: false
      retain:
        default: 21

    rtmp:
      enabled: false

    timestamp_style:
      position: "bl"
      thickness: 2

    best_image_timeout: 30

    zones:  
      zone_front_porch:
        coordinates: 858,1529,613,1615,561,1675,363,1669,41,1920,636,1920,2237,1920,1996,1577,1662,1618,1590,1814,1455,1825,1370,1721,1176,1694,1205,1557

      zone_front_sidewalk:
        coordinates: 281,1205,758,1273,1023,1296,1366,1305,1879,1276,2267,1230,2197,1217,2029,1241,1636,1271,1197,1271,852,1253,664,1236,459,1208,288,1179

      zone_driveway:
        coordinates: 2326,1471,2455,1246,2303,1241,1834,1313

      zone_front_lawn:
        coordinates: 1725,1297,1631,1314,1248,1330,843,1322,255,1215,199,1314,173,1442,377,1410,840,1503,1154,1527,1351,1554,1467,1463,1605,1465,1674,1559,2221,1470

################### SIDEDRIVE
  Sidedrive:
    ffmpeg:
      inputs:
        - path: rtsp://10.X.X.4/live0
          roles:
            - detect
            - record
      global_args: 
        -hide_banner 
        -loglevel error
      # hwaccel_args: 
        # - -hwaccel
        # - vaapi
        # - -hwaccel_device
        # - /dev/dri/renderD128
        # - -hwaccel_output_format
        # - yuv420p
      # input_args: 
        # -avoid_negative_ts make_zero 
        # -fflags +genpts+discardcorrupt 
        # -rtsp_transport tcp 
        # -stimeout 5000000 
        # -use_wallclock_as_timestamps 1
      output_args:
        record: 
          -f segment 
          -segment_time 10 
          -segment_format mp4 
          -reset_timestamps 1 
          -strftime 1 
          -c copy 
          #-an

    detect:
      enabled: true
      max_disappeared: 30
      width: 1920
      height: 1080
      fps: 5

    objects: # Optional: Camera level object filters config.
      track:
        - person
        - dog
      filters:
        person:
          min_area: 1500
          max_area: 200000
          min_score: 0.7
          threshold: 0.74
        dog:
          min_area: 3500
          max_area: 50000
          min_score: 0.65
          threshold: 0.70

    motion:
      threshold: 25
      improve_contrast: true
      mask:
        - 1920,0,1920,60,1643,56,1640,0
        - 0,791,27,792,106,408,242,171,0,0

    record:
      enabled: true
      retain:
        days: 10
      events:
        #max_seconds: 300
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - dog
        retain:
          default: 14

    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      crop: false
      retain:
        default: 14

    rtmp:
      enabled: false

    timestamp_style:
      position: "bl"
      thickness: 2

    best_image_timeout: 30

################### SIDEYARD
  # Sideyard:
    # ffmpeg:
      # inputs:
        # - path: rtsp://10.X.X.2/live0
          # roles:
            # - detect
            # - record
      # global_args: 
        # -hide_banner 
        # -loglevel error
      # hwaccel_args: 
        # - -hwaccel
        # - vaapi
        # - -hwaccel_device
        # - /dev/dri/renderD128
        # - -hwaccel_output_format
        # - yuv420p
      # input_args: 
        # -avoid_negative_ts make_zero 
        # -fflags +genpts+discardcorrupt 
        # -rtsp_transport tcp 
        # -stimeout 5000000 
        # -use_wallclock_as_timestamps 1
      # output_args:
        # record: 
          # -f segment 
          # -segment_time 10 
          # -segment_format mp4 
          # -reset_timestamps 1 
          # -strftime 1 
          # -c copy 
          # -an

    # detect:
      # enabled: true
      # max_disappeared: 30
      # width: 1920
      # height: 1080
      # fps: 5

    # objects:
      # track:
        # - person
        # - dog
        # - car
      # filters:
        # person:
          # min_area: 5000
          # max_area: 110000
          # min_score: 0.7
          # threshold: 0.74
        # dog:
          # min_area: 1000
          # max_area: 15000
          # min_score: 0.65
          # threshold: 0.70
          # mask:
            # - 1385,105,1418,362,1167,319,1166,140
        # car:
          # min_area: 2500
          # max_area: 30000
          # min_score: 0.65
          # threshold: 0.70

    # motion:
      # threshold: 25
      # improve_contrast: true
      # mask:
        # - 1920,0,1920,410,1812,480,1643,56,1640,0
        # - 912,1080,1152,1080,1282,327,1244,293,1167,337
        # - 427,828,477,1018,851,905,1301,729,1564,601,1477,501,1094,617,821,684

    # record:
      # enabled: true
      # retain:
        # days: 5
        # mode: all
      # events:
        # max_seconds: 300
        # pre_capture: 5
        # post_capture: 5
        # objects:
          # - person
          # - dog
          # - car
        # retain:
          # default: 7

    # snapshots:
      # enabled: true
      # timestamp: true
      # bounding_box: true 
      # crop: false
      # retain:
        # default: 7

    # rtmp:
      # enabled: false

    # timestamp_style:
      # position: "bl"
      # thickness: 2

    # best_image_timeout: 30

################### BACKYARD
  Backyard:
    ffmpeg:
      inputs:
        #- path: rtsp://10.X.X.44/live0 
        - path: rtsp://10.X.X.6/live0
          roles:
            - detect
            - record
      global_args: 
        -hide_banner 
        -loglevel error
      # hwaccel_args: 
        # - -hwaccel
        # - vaapi
        # - -hwaccel_device
        # - /dev/dri/renderD128
        # - -hwaccel_output_format
        # - yuv420p
      # input_args: 
        # -avoid_negative_ts make_zero 
        # -fflags +genpts+discardcorrupt 
        # -rtsp_transport tcp 
        # -stimeout 5000000 
        # -use_wallclock_as_timestamps 1
      output_args:
        record: 
          -f segment 
          -segment_time 10 
          -segment_format mp4 
          -reset_timestamps 1 
          -strftime 1 
          -c copy 
          #-an

    detect:
      enabled: true
      max_disappeared: 30
      width: 1920
      height: 1080
      fps: 5

    objects: # Optional: Camera level object filters config.
      track:
        - person
        - dog
      filters:
        person:
          min_area: 16000
          max_area: 165000
          min_score: 0.72
          threshold: 0.78
        dog:
          min_area: 2000
          max_area: 50000
          min_score: 0.55
          threshold: 0.70

    motion:
      threshold: 22 #lower higher sensitivity; higher lower sensitivity
      improve_contrast: true
      mask:
        - 1236,0,1090,0,1120,90,1231,114,1397,126,1452,129,1495,84,1547,75,1619,91,1660,150,1731,128,1920,147,1920,0
        - 922,85,1114,67,1084,0,911,0
        - 778,0,767,67,699,40,662,88,486,120,362,100,244,100,216,148,89,171,0,291,0,162,0,0,287,0

    record:
      enabled: true
      retain:
        days: 7
      events:
        #max_seconds: 600
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - dog
        retain:
          default: 14

    snapshots:
      enabled: true
      timestamp: true 
      bounding_box: true
      crop: false
      retain:
        default: 14

    rtmp:
      enabled: false

    timestamp_style:
      position: "bl"
      thickness: 2

    best_image_timeout: 30

################### GARAGE
  Garage:
    ffmpeg:
      inputs:
        - path: rtsp://X:X@10.X.X.5
        #- path: rtsp://X:X@10.X.X.5/live0
          roles:
            - detect
            - record
      global_args: 
        -hide_banner 
        -loglevel error
      # hwaccel_args: 
        # - -hwaccel
        # - vaapi
        # - -hwaccel_device
        # - /dev/dri/renderD128
        # - -hwaccel_output_format
        # - yuv420p
      # input_args: 
        # -avoid_negative_ts make_zero 
        # -fflags +genpts+discardcorrupt 
        # -rtsp_transport tcp 
        # -stimeout 5000000 
        # -use_wallclock_as_timestamps 1
      output_args:
        record: 
          -f segment 
          -segment_time 10 
          -segment_format mp4 
          -reset_timestamps 1 
          -strftime 1 
          -c copy 
          #-an

    detect:
      enabled: true
      max_disappeared: 30
      width: 1280
      height: 960
      fps: 5

    objects:
      track:
        - person
        - dog
      filters:
        person:
          min_area: 1100
          max_area: 400000
          min_score: 0.70
          threshold: 0.74
        dog:
          min_area: 1000
          max_area: 50000
          min_score: 0.60
          threshold: 0.70
          mask:
            - 1435,148,1876,285,1684,731,1316,422

    motion:
      threshold: 25
      improve_contrast: false
      mask:
        - 160,85,649,77,892,85,996,48,1003,0,0,0,0,146
        - 1920,0,1920,206,1725,234,1513,153,1255,94,1213,0

    record:
      enabled: true
      retain:
        days: 0
      events:
        #max_seconds: 300
        pre_capture: 5
        post_capture: 5
        objects:
          - person
          - dog
        retain:
          default: 7

    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      crop: false
      retain:
        default: 7

    rtmp:
      enabled: false

    timestamp_style:
      position: "bl"
      thickness: 2

    best_image_timeout: 30

Relevant log output

There is no output for my wired cams (Driveway and Garage), but my wireless ones have the usual ffmpeg errors due to dropped frames

FFprobe output from your camera

ffprobe version 4.4.1-Jellyfin Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-static --enable-gmp --enable-gnutls --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --arch=amd64 --enable-opencl --enable-vaapi --enable-amf --enable-libmfx --enable-vdpau --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from 'rtsp://X:X@10.0.30.3':
  Metadata:
    title           : Media Server
  Duration: N/A, start: 0.129000, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 2048x1536, 30 fps, 100 tbr, 90k tbn, 60 tbc
  Stream #0:1: Audio: aac (LC), 8000 Hz, mono, fltp
root@frigatelxc:/opt/frigate#

Frigate stats

{"Backyard":{"camera_fps":5.0,"capture_pid":246,"detection_fps":12.7,"pid":232,"process_fps":5.1,"skipped_fps":0.0},"Doorbell":{"camera_fps":5.0,"capture_pid":236,"detection_fps":8.1,"pid":230,"process_fps":5.0,"skipped_fps":0.0},"Driveway":{"camera_fps":5.0,"capture_pid":235,"detection_fps":5.8,"pid":227,"process_fps":5.0,"skipped_fps":0.0},"Garage":{"camera_fps":5.0,"capture_pid":252,"detection_fps":0.0,"pid":233,"process_fps":5.0,"skipped_fps":0.0},"Sidedrive":{"camera_fps":5.6,"capture_pid":240,"detection_fps":8.2,"pid":231,"process_fps":4.9,"skipped_fps":0.0},"detection_fps":34.8,"detectors":{"coral":{"detection_start":0.0,"inference_speed":11.25,"pid":216}},"service":{"latest_version":"0.10.1","storage":{"/dev/shm":{"free":243.1,"mount_type":"tmpfs","total":268.4,"used":25.3},"/media/frigate/clips":{"free":876578.6,"mount_type":"ext4","total":1051682.2,"used":132739.5},"/media/frigate/recordings":{"free":876578.6,"mount_type":"ext4","total":1051682.2,"used":132739.5},"/tmp/cache":{"free":979.5,"mount_type":"tmpfs","total":1000.0,"used":20.5}},"temperatures":{},"uptime":3937,"version":"0.11.0-d2c3cdc"}}

Operating system

Proxmox

Install method

Docker Compose

Coral version

USB

Network connection

Mixed

Camera make and model

Amcrest IP5M-T1179EW

Any other information that may be helpful

No response

NickM-27 commented 2 years ago

Yeah, the recommended reolink args are different than the ones that would be good for amcrest. In my experience with my amcrest cams I have not needed any specific args (except to enable audio).

If there are no logs for those cameras than it is unclear what the issue could be. Might be worth trying what was recommended here and duplicate the inputs with one as record and one as detect just to see if perhaps running in separate ffmpeg processes fixes the issue

esmoyer commented 2 years ago

Thanks, I'll give it a go and keep testing things out. It's not terrible, just annoying that I can't seem to figure it out. I'm planning to de-virtualize Frigate and put it on it's own HP Pro mini once I get a retired one from work to see if that fixes any of the few wonky bits I have encountered.

NickM-27 commented 2 years ago

I'm planning to de-virtualize Frigate and put it on it's own HP Pro mini once I get a retired one from work to see if that fixes any of the few wonky bits I have encountered.

That could be an issue too, perhaps cpu from detect decoding and motion detection is hitting it too hard. Separate ffmpeg processes should help this if that's the case

esmoyer commented 2 years ago

Interesting. I haven't seen any crazy CPU spikes but may watch a bit more. Just curious, what's your setup? Do you have a dedicated machine or do you virtualize? If so, what specs are you sitting at?

NickM-27 commented 2 years ago

Interesting. I haven't seen any crazy CPU spikes but may watch a bit more.

This wasn't a solid thought, more of a random guess.

Just curious, what's your setup? Do you have a dedicated machine or do you virtualize? If so, what specs are you sitting at?

I run in UnRAID so directly on docker as a dedicated server. I have a ryzen 5700G and 32 GB RAM writing to NVME SSDs. To be clear it's not that I never get a random stutter but it is very rare.

Also another thing I thought of that I would recommend trying, if you download the mp4 from a frigate event and view it in VLC do you still see the stutter? There is a known issue with the viewer inside frigate stuttering that is unrelated to the actual footage stored

esmoyer commented 2 years ago

Also another thing I thought of that I would recommend trying, if you download the mp4 from a frigate event and view it in VLC do you still see the stutter? There is a known issue with the viewer inside frigate stuttering that is unrelated to the actual footage stored

Good call. Just tried it with a fresh clip. Playing in Frigate yielded a pretty smooth video but a 2-3 second pause while the downloaded MP4 was smooth all the way. I'll test it out a bit more. Thinking about it, I previously thought it had to do with it stitching together the 10 second clips as the pauses/stutters seemed to happen a 8-10 second intervals; but figured correlation does not always equal causation.