search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.08k stars 1.74k forks source link

[Support]: Multiple errors after 21.5 hours of clean start (delete old video and DB) as follows #6771

Closed ankycooper closed 1 year ago

ankycooper commented 1 year ago

Describe the problem you are having

Multiple errors after 21.5 hours of clean start (delete old video and DB) as follows 

[swscaler @ 0x5618140432c0] [swscaler @ 0x561814052540] deprecated pixel format used, make sure you did set range correctly

2023-06-09 16:04:21.637680477  [2023-06-09 16:04:21] ffmpeg.Shop.record             ERROR   : [rtsp @ 0x55a0ca92ad40] RTP: PT=60: bad cseq f246 expected=e011
2023-06-09 16:04:21.638439942  [2023-06-09 16:04:21] ffmpeg.Shop.record             ERROR   : rtsp://127.0.0.1:8554/Shop: Connection timed out
2023-06-09 16:04:21.659071427  [2023-06-09 16:04:21] watchdog.Shop                  INFO    : Terminating the existing ffmpeg process...
2023-06-09 16:04:21.660483611  [2023-06-09 16:04:21] watchdog.Shop                  INFO    : Waiting for ffmpeg to exit gracefully...
2023-06-09 16:04:21.810062471  [2023-06-09 16:04:21] ffmpeg.Wing-Door.record        ERROR   : [segment @ 0x55a50a01f880] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2023-06-09 16:04:21.810317984  [2023-06-09 16:04:21] ffmpeg.Wing-Door.record        ERROR   : [segment @ 0x55a50a01f880] Non-monotonous DTS in output stream 0:0; previous: 2290698900, current: 2290698900; changing to 2290698901. This may result in incorrect 

Unable to keep up with recording segments in cache for Backyard-Right. Keeping the 5 most recent segments out of 6 and discarding the rest...

 Failed to probe corrupt segment /tmp/cache/Outdoor-Room-20230610132639.mp4: 0 - b'[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5632877f66c0] moov atom not found\n/tmp/cache/Outdoor-Room-20230610132639.mp4: Invalid data found when processing input\n'

Version

0.12.0-DA3E197

Frigate config file

mqtt:
  host: MQ-SVR
  user: mq-user
  password: mq-pass

detectors:
  coral_pci:
    type: edgetpu
    device: pci 
  # coral_usb:
  #   type: edgetpu
  #   device: usb 

# logger:
#   # Optional: default log level (default: shown below)
#   default: debug

go2rtc:
  rtsp:
    username: "go2rtc-admin"
    password: "go2rtc-pass"

  webrtc:
    candidates:
      - 10.100.100.9:8555
      - stun:8555

  streams:
    Wing-Door: #<------ Name the camera
      - rtsp://cam-user:cam-pass@10.30.30.2:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    Wing-Door-Sub: #<------ Name the camera substream
      - rtsp://cam-user:cam-pass@10.30.30.2:554/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif
      - "ffmpeg:Wing-Door-Sub#audio=opus"
    House-Front: #<------ Name the camera
      - rtsp://cam-user:cam-pass@10.30.30.3:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    House-Front-Sub: #<------ Name the camera substream
      - rtsp://cam-user:cam-pass@10.30.30.3:554/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif
    Garage-Front: #<------ Name the camera
      - rtsp://cam-user:cam-pass@10.30.30.4:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    Garage-Front-Sub: #<------ Name the camera substream
      - rtsp://cam-user:cam-pass@10.30.30.4:554/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif
    Backyard-Left: #<------ Name the camera
      - rtsp://cam-user:cam-pass@10.30.30.5:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    Backyard-Left-Sub: #<------ Name the camera substream
      - rtsp://cam-user:cam-pass@10.30.30.5:554/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif
    Wing-Door-Back: #<------ Name the camera
      - rtsp://cam-user:cam-pass@10.30.30.6:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    Wing-Door-Back-Sub: #<------ Name the camera substream
      - rtsp://cam-user:cam-pass@10.30.30.6:554/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif
    Backyard-Right: #<------ Name the camera
      - rtsp://cam-user:cam-pass@10.30.30.7:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    Backyard-Right-Sub: #<------ Name the camera substream
      - rtsp://cam-user:cam-pass@10.30.30.7:554/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif
    Shop: #<------ Name the camera
      - rtsp://cam-user:cam-pass@10.30.30.8:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    Shop-Sub: #<------ Name the camera substream
      - rtsp://cam-user:cam-pass@10.30.30.8:554/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif
    Staircase: #<------ Name the camera
      - rtsp://cam-user:cam-pass@10.30.30.9:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
    Staircase-Sub: #<------ Name the camera substream
      - rtsp://cam-user:cam-pass@10.30.30.9:554/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif
    Outdoor-Room: #<------ Name the camera
      - rtsp://admin:abcd4321A@10.30.30.10:554/H264/ch1/main/av_stream
    Outdoor-Room-Sub: #<------ Name the camera substream
      - rtsp://admin:abcd4321A@10.30.30.10:554/H264/ch1/sub/av_stream

# ffmpeg:
#   hwaccel_args: preset-vaapi   

ui:
  use_experimental: false
  live_mode: mse

database:
  # The path to store the SQLite DB (default: shown below)
  path: /config/frigate.db

# timestamp_style:
#   # Optional: Position of the timestamp (default: shown below)
#   #           "tl" (top left), "tr" (top right), "bl" (bottom left), "br" (bottom right)
#   position: "tl"
#   # Optional: Format specifier conform to the Python package "datetime" (default: shown below)
#   #           Additional Examples:
#   #             german: "%d.%m.%Y %H:%M:%S"
#   format: "%m/%d/%Y %H:%M:%S"
#   # Optional: Color of font
#   color:
#     # All Required when color is specified (default: shown below)
#     red: 255
#     green: 255
#     blue: 255
#   # Optional: Line thickness of font (default: shown below)
#   thickness: 2
#   # Optional: Effect of lettering (default: shown below)
#   #           None (No effect),
#   #           "solid" (solid background in inverse color of font)
#   #           "shadow" (shadow for font)
#   effect: shadow
rtmp:
  enabled: false

birdseye:
  enabled: True
  restream: False
  width: 1280 
  height: 720
  quality: 1
  mode: objects

# live:
#   height: 640
#   quality: 1

snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 7

detect:
  width: 1280 # <---- update for your camera's resolution
  height: 720 # <---- update for your camera's resolution
  fps: 5
objects:
  track:
      - person
      - dog
      # - car
      # - cat
  filters:
    person:
      min_score: 0.4
      threshold: 0.6
      min_area: 700

record:
  enabled: True
  retain: 
    days: 2 
    mode: all #<--- 27/7 recording retained for 2 days 
  events:
    retain:
      default: 7 
      mode: motion  #<---- Motion events retained for 5 days other option is active_objects
    pre_capture: 3
    post_capture: 10

cameras:
  Wing-Door: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Wing-Door-Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Wing-Door
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-copy
    motion:
      mask:
        - 523,246,1015,175,1105,200,1025,720,1280,720,1280,0,0,0,0,357
        - 923,720,1146,720,1196,505,1127,232,1041,458,972,628
  House-Front: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/House-Front-Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/House-Front
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-copy
    objects:
      track:
          - person
          - dog
          - car
          - cat
      filters:
        person:
          min_score: 0.4
          threshold: 0.6
          min_area: 700
    motion:
      mask:
      - 1280,165,980,139,977,75,1024,55,1017,0,1280,0
      - 297,191,691,103,771,178,809,140,826,38,824,26,800,0,0,0,0,326
    zones:
      front-porch:
        coordinates: 1280,720,0,720,0,542,42,523,260,410,580,361,1196,453
      nature-strip:
        coordinates: 570,221,762,246,1093,283,1235,307,1219,406,577,299
      road:
        coordinates: 1236,293,1257,0,550,0,564,216
  Garage-Front: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Garage-Front-Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Garage-Front
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-copy
    motion:
      mask:
        - 523,246,1015,175,1105,200,1025,720,1280,720,1280,0,0,0,0,357
        - 923,720,1146,720,1196,505,1127,232,1041,458,972,628
    zones:
      road:
        coordinates: 43,190,59,363,248,317,392,293,475,283,473,248,602,167,767,178,776,0,48,0
      nature-strip:
        coordinates: 71,440,298,367,434,333,459,285,269,318,57,365
      driveway:
        coordinates: 419,333,542,419,735,435,821,464,899,486,1162,552,1199,720,158,720,72,448
      front-porch:
        coordinates: 919,490,954,309,932,292,884,279,865,377,830,462

  Backyard-Left: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Backyard-Left-Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Backyard-Left
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-copy
    motion:
      mask:
        - 0,720,39,318,75,179,429,160,843,170,939,221,1227,376,1280,414,1280,0,193,0,50,0,0,0,0,146,0,242,0,363,0,416,0,597,42,720,0,597,0,720
        - 31,699,215,699,215,667,31,670

  Wing-Door-Back: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Wing-Door-Back-Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Wing-Door-Back
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-copy
    motion:
      mask:
        - 398,0,1280,0,1280,720,621,720,590,562,558,340,464,288,370,276
        - 0,720,233,720,211,448,370,253,391,108,416,0,0,0

  Backyard-Right: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Backyard-Right-Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Backyard-Right
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-copy
    motion:
      mask:
        - 0,720,238,720,156,549,121,316,121,155,135,0,0,0
        - 888,231,1280,325,1280,0,127,0,122,94,113,299,570,177

  Shop: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Shop-Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Shop
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-copy
    motion:
      mask:
        - 809,0,830,21,862,34,948,57,1002,69,1175,129,1280,176,1280,98,1280,0
        - 0,0,0,720,102,720,51,377,90,69,169,52,162,0
        - 938,720,1280,720,1280,203

  Staircase: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Staircase-Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Staircase
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-copy
    motion:
      mask:
        - 1280,720,958,720,937,661,892,497,881,418,893,366,906,309,924,166,934,71,901,59,652,51,538,21,0,63,0,0,1280,0
        - 340,306,331,222,272,214,241,141,156,143,132,0,0,0,0,501
  Outdoor-Room: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Outdoor-Room-Sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/Outdoor-Room
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-copy
    detect:
      width: 640 # <---- update for your camera's resolution
      height: 360 # <---- update for your camera's resolution
      fps: 6
    motion:
      mask:
        - 299,0,205,152,262,360,0,360,0,0
        - 640,202,640,0,463,0

Relevant log output

Summmerised logs 
2023-06-09 13:57:06.578040019  [2023-06-09 13:57:06] watchdog.Outdoor-Room          INFO    : Outdoor-Room exceeded fps limit. Exiting ffmpeg...
2023-06-09 13:57:06.582926866  [2023-06-09 13:57:06] watchdog.Outdoor-Room          INFO    : Waiting for ffmpeg to exit gracefully...
2023-06-09 13:57:10.155123006  [2023-06-09 13:57:10] ffmpeg.House-Front.record      ERROR   : [segment @ 0x55cd4e3e2300] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly

2023-06-10 13:16:59.188193147  [2023-06-10 13:16:59] ffmpeg.Wing-Door.record        ERROR   : [segment @ 0x556712a91c40] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
2023-06-10 13:16:59.188206554  [2023-06-10 13:16:59] ffmpeg.Wing-Door.record        ERROR   : [segment @ 0x556712a91c40] Non-monotonous DTS in output stream 0:0; previous: 1794600, current: 1794600; changing to 1794601. This may result in incorrect timestamps in the output file.
2023-06-10 13:16:59.188213030  [2023-06-10 13:16:59] ffmpeg.Wing-Door.record        ERROR   : [segment @ 0x556712a91c40] Non-monotonous DTS in output stream 0:0; previous: 1794601, current: 1794600; changing to 1794602. This may result in incorrect timestamps in the output file.
2023-06-10 13:16:59.188219297  [2023-06-10 13:16:59] ffmpeg.Wing-Door.record        ERROR   : [segment @ 0x556712a91c40] Non-monotonous DTS in output stream 0:0; previous: 1795500, current: 1795500; changing to 1795501. This may result in incorrect timestamps in the output file.
2023-06-10 13:16:59.188225244  [2023-06-10 13:16:59] ffmpeg.Wing-Door.record        ERROR   : [segment @ 0x556712a91c40] Non-monotonous DTS in output stream 0:0; previous: 1795501, current: 1795500; changing to 1795502. This may result in incorrect timestamps in the output file.

2023-06-09 13:57:10.176617349  [2023-06-09 13:57:10] ffmpeg.House-Front.record      ERROR   : rtsp://127.0.0.1:8554/House-Front: Connection timed out
2023-06-09 13:57:10.177539676  [2023-06-09 13:57:10] watchdog.House-Front           INFO    : Terminating the existing ffmpeg process...
2023-06-09 13:57:10.178396607  [2023-06-09 13:57:10] watchdog.House-Front           INFO    : Waiting for ffmpeg to exit gracefully...
2023-06-09 13:57:40.967517463  [2023-06-09 13:57:40] watchdog.Outdoor-Room          INFO    : FFmpeg did not exit. Force killing...
2023-06-09 13:57:40.971064041  [2023-06-09 13:57:40] frigate.video                  ERROR   : Outdoor-Room: Unable to read frames from ffmpeg process.
2023-06-09 13:57:40.972602718  [2023-06-09 13:57:40] frigate.video                  ERROR   : Outdoor-Room: ffmpeg process is not running. exiting capture thread...
2023-06-09 13:57:55.281954245  [2023-06-09 13:57:55] watchdog.Outdoor-Room          ERROR   : Ffmpeg process crashed unexpectedly for Outdoor-Room.
2023-06-09 13:57:55.281967872  [2023-06-09 13:57:55] watchdog.Outdoor-Room          ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-06-09 13:57:55.281974612  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [swscaler @ 0x5618140432c0] [swscaler @ 0x561814052540] deprecated pixel format used, make sure you did set range correctly
2023-06-09 13:57:55.282009628  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [swscaler @ 0x5618140432c0] [swscaler @ 0x561814006d40] deprecated pixel format used, make sure you did set range correctly
2023-06-09 13:57:55.282072492  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [swscaler @ 0x561814016e40] [swscaler @ 0x561814052540] deprecated pixel format used, make sure you did set range correctly
2023-06-09 13:57:55.282144219  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [swscaler @ 0x561814016e40] [swscaler @ 0x561814043e40] deprecated pixel format used, make sure you did set range correctly
2023-06-09 13:57:55.282155416  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [swscaler @ 0x561814016e40] [swscaler @ 0x561814052540] deprecated pixel format used, make sure you did set range correctly
2023-06-09 13:57:55.282210819  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [swscaler @ 0x561814016e40] [swscaler @ 0x5618140432c0] deprecated pixel format used, make sure you did set range correctly
2023-06-09 13:57:55.282218899  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [swscaler @ 0x561814016e40] [swscaler @ 0x561814052540] deprecated pixel format used, make sure you did set range correctly
2023-06-09 13:57:55.282277272  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [swscaler @ 0x561814016e40] [swscaler @ 0x5618140432c0] deprecated pixel format used, make sure you did set range correctly
2023-06-09 13:57:55.282283472  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f60240] Increasing reorder buffer to 4
2023-06-09 13:57:55.282289048  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f95680] Invalid NAL unit 0, skipping.
2023-06-09 13:57:55.282294505  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f60240] Invalid NAL unit 0, skipping.
2023-06-09 13:57:55.282366549  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f95680] Invalid NAL unit 0, skipping.
2023-06-09 13:57:55.282372932  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f60240] Invalid NAL unit 0, skipping.
2023-06-09 13:57:55.282377956  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   :     Last message repeated 1 times
2023-06-09 13:57:55.282383685  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f95680] Invalid NAL unit 0, skipping.
2023-06-09 13:57:55.282388896  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f60240] Invalid NAL unit 0, skipping.
2023-06-09 13:57:55.282473732  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   :     Last message repeated 2 times
2023-06-09 13:57:55.282480602  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f95680] Invalid NAL unit 0, skipping.
2023-06-09 13:57:55.282534445  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f60240] Invalid NAL unit 0, skipping.
2023-06-09 13:57:55.283024789  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f95680] Invalid NAL unit 0, skipping.
2023-06-09 13:57:55.283410726  [2023-06-09 13:57:55] ffmpeg.Outdoor-Room.detect     ERROR   : [h264 @ 0x561813f60240] Invalid NAL unit 0, skipping.

2023-06-10 13:21:34.599286216  [2023-06-10 13:21:34] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Backyard-Left. Keeping the 5 most recent segments out of 24 and discarding the rest...
2023-06-10 13:21:34.657038162  [2023-06-10 13:21:34] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Backyard-Right. Keeping the 5 most recent segments out of 24 and discarding the rest...
2023-06-10 13:21:34.705885186  [2023-06-10 13:21:34] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Garage-Front. Keeping the 5 most recent segments out of 24 and discarding the rest...
2023-06-10 13:21:34.760610797  [2023-06-10 13:21:34] frigate.record                 WARNING : Unable to keep up with recording segments in cache for House-Front. Keeping the 5 most recent segments out of 24 and discarding the rest...
2023-06-10 13:21:34.803885084  [2023-06-10 13:21:34] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Outdoor-Room. Keeping the 5 most recent segments out of 23 and discarding the rest...
2023-06-10 13:21:34.819162286  [2023-06-10 13:21:34] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Shop. Keeping the 5 most recent segments out of 24 and discarding the rest...
2023-06-10 13:21:34.866790441  [2023-06-10 13:21:34] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Staircase. Keeping the 5 most recent segments out of 24 and discarding the rest...
2023-06-10 13:21:34.913262063  [2023-06-10 13:21:34] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Wing-Door. Keeping the 5 most recent segments out of 24 and discarding the rest...
2023-06-10 13:21:34.969066917  [2023-06-10 13:21:34] frigate.record                 WARNING : Unable to keep up with recording segments in cache for Wing-Door-Back. Keeping the 5 most recent segments out of 24 and discarding the rest...
2023-06-10 13:21:35.100178273  [2023-06-10 13:21:35] frigate.record                 WARNING : Failed to probe corrupt segment /tmp/cache/Backyard-Left-20230610132051.mp4: 0 - b'[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55e525e4c6c0] moov atom not found\n/tmp/cache/Backyard-Left-20230610132051.mp4: Invalid data found when processing input\n'
2023-06-10 13:21:35.100521005  [2023-06-10 13:21:35] frigate.record                 WARNING : Discarding a corrupt recording segment: /tmp/cache/Backyard-Left-20230610132051.mp4
2023-06-10 13:21:35.184765012  [2023-06-10 13:21:35] frigate.record                 WARNING : Failed to probe corrupt segment /tmp/cache/Backyard-Left-20230610132101.mp4: 0 - b'[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5607f5b2c6c0] moov atom not found\n/tmp/cache/Backyard-Left-20230610132101.mp4: Invalid data found when processing input\n'

2023-06-10 13:32:30.831907160  [2023-06-10 13:32:30] ffmpeg.Staircase.record        ERROR   : av_interleaved_write_frame(): No space left on device
2023-06-10 13:32:30.831919216  [2023-06-10 13:32:30] ffmpeg.Staircase.record        ERROR   : [segment @ 0x561bfb0839c0] Failure occurred when ending segment '/tmp/cache/Staircase-20230610133223.mp4'
2023-06-10 13:32:30.832326301  [2023-06-10 13:32:30] ffmpeg.Staircase.record        ERROR   : Error writing trailer of /tmp/cache/Staircase-%Y%m%d%H%M%S.mp4: No space left on device
2023-06-10 13:32:30.832716978  [2023-06-10 13:32:30] watchdog.Staircase             INFO    : Terminating the existing ffmpeg process...
2023-06-10 13:32:30.833194655  [2023-06-10 13:32:30] watchdog.Staircase             INFO    : Waiting for ffmpeg to exit gracefully...
2023-06-10 13:32:36.308249784  [2023-06-10 13:32:36] ffmpeg.Garage-Front.record     ERROR   : [segment @ 0x55bf5c5c6200] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly

FFprobe output from your camera

NA

Frigate stats

{"Backyard-Left":{"camera_fps":5.1,"capture_pid":479,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":621746,"pid":456,"process_fps":5.1,"skipped_fps":0.0},"Backyard-Right":{"camera_fps":5.1,"capture_pid":506,"detection_enabled":1,"detection_fps":5.5,"ffmpeg_pid":621654,"pid":460,"process_fps":5.1,"skipped_fps":0.0},"Garage-Front":{"camera_fps":5.0,"capture_pid":473,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":621749,"pid":444,"process_fps":5.0,"skipped_fps":0.0},"House-Front":{"camera_fps":5.0,"capture_pid":468,"detection_enabled":1,"detection_fps":0.9,"ffmpeg_pid":621652,"pid":443,"process_fps":5.0,"skipped_fps":0.0},"Outdoor-Room":{"camera_fps":6.0,"capture_pid":529,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":632470,"pid":464,"process_fps":6.0,"skipped_fps":0.0},"Shop":{"camera_fps":5.0,"capture_pid":516,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":621656,"pid":461,"process_fps":5.0,"skipped_fps":0.0},"Staircase":{"camera_fps":5.0,"capture_pid":523,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":621655,"pid":463,"process_fps":5.0,"skipped_fps":0.0},"Wing-Door":{"camera_fps":5.0,"capture_pid":466,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":621747,"pid":439,"process_fps":5.0,"skipped_fps":0.0},"Wing-Door-Back":{"camera_fps":5.0,"capture_pid":495,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":621684,"pid":458,"process_fps":5.0,"skipped_fps":0.0},"cpu_usages":{"%Cpu(s):":{"cpu":"id,","mem":"0.3"},"1":{"cpu":"0.0","mem":"0.0"},"101":{"cpu":"17.4","mem":"4.7"},"117":{"cpu":"0.0","mem":"0.1"},"134":{"cpu":"0.0","mem":"0.5"},"135":{"cpu":"0.0","mem":"0.2"},"136":{"cpu":"0.0","mem":"0.1"},"137":{"cpu":"0.0","mem":"0.1"},"140":{"cpu":"0.0","mem":"0.1"},"15":{"cpu":"0.0","mem":"0.0"},"152":{"cpu":"0.0","mem":"0.4"},"16":{"cpu":"0.0","mem":"0.0"},"172":{"cpu":"0.0","mem":"0.1"},"24":{"cpu":"0.0","mem":"0.0"},"25":{"cpu":"0.0","mem":"0.0"},"26":{"cpu":"0.0","mem":"0.0"},"27":{"cpu":"0.0","mem":"0.0"},"28":{"cpu":"0.0","mem":"0.0"},"29":{"cpu":"0.0","mem":"0.0"},"30":{"cpu":"0.0","mem":"0.0"},"307":{"cpu":"0.0","mem":"0.1"},"31":{"cpu":"0.0","mem":"0.0"},"32":{"cpu":"0.0","mem":"0.0"},"41":{"cpu":"0.0","mem":"0.0"},"42":{"cpu":"0.0","mem":"0.0"},"425":{"cpu":"0.0","mem":"1.0"},"431":{"cpu":"1.6","mem":"0.1"},"433":{"cpu":"7.2","mem":"1.2"},"438":{"cpu":"0.0","mem":"0.1"},"439":{"cpu":"1.6","mem":"1.3"},"442":{"cpu":"0.0","mem":"0.2"},"443":{"cpu":"3.9","mem":"1.5"},"444":{"cpu":"2.3","mem":"1.3"},"445":{"cpu":"0.0","mem":"0.1"},"456":{"cpu":"1.6","mem":"1.4"},"457":{"cpu":"0.0","mem":"0.1"},"458":{"cpu":"1.6","mem":"1.2"},"459":{"cpu":"0.0","mem":"0.1"},"460":{"cpu":"21.6","mem":"1.4"},"461":{"cpu":"2.0","mem":"1.4"},"462":{"cpu":"0.0","mem":"0.1"},"463":{"cpu":"1.6","mem":"1.3"},"464":{"cpu":"2.0","mem":"1.2"},"465":{"cpu":"0.0","mem":"0.2"},"466":{"cpu":"4.9","mem":"1.1"},"467":{"cpu":"0.0","mem":"0.1"},"468":{"cpu":"4.3","mem":"1.2"},"471":{"cpu":"0.0","mem":"0.1"},"473":{"cpu":"4.6","mem":"1.2"},"478":{"cpu":"0.0","mem":"0.2"},"479":{"cpu":"3.9","mem":"1.1"},"495":{"cpu":"5.2","mem":"1.2"},"506":{"cpu":"5.2","mem":"1.2"},"516":{"cpu":"4.3","mem":"1.2"},"523":{"cpu":"4.6","mem":"1.2"},"529":{"cpu":"2.3","mem":"14.0"},"621652":{"cpu":"12.5","mem":"0.4"},"621654":{"cpu":"15.1","mem":"0.5"},"621655":{"cpu":"19.0","mem":"0.4"},"621656":{"cpu":"15.7","mem":"0.5"},"621665":{"cpu":"3.3","mem":"3.6"},"621684":{"cpu":"15.4","mem":"0.5"},"621746":{"cpu":"12.1","mem":"0.5"},"621747":{"cpu":"19.7","mem":"0.5"},"621749":{"cpu":"19.0","mem":"0.4"},"621944":{"cpu":"3.9","mem":"0.3"},"621946":{"cpu":"4.9","mem":"0.3"},"621947":{"cpu":"4.3","mem":"0.3"},"621948":{"cpu":"4.6","mem":"0.3"},"621949":{"cpu":"5.6","mem":"0.3"},"621950":{"cpu":"3.9","mem":"0.3"},"621951":{"cpu":"3.9","mem":"0.3"},"621952":{"cpu":"4.6","mem":"0.3"},"632396":{"cpu":"0.3","mem":"0.2"},"632470":{"cpu":"6.6","mem":"0.4"},"724025":{"cpu":"0.0","mem":"0.0"},"724083":{"cpu":"0.0","mem":"0.0"},"724094":{"cpu":"0.3","mem":"0.0"},"80":{"cpu":"0.0","mem":"0.0"},"81":{"cpu":"0.0","mem":"0.0"},"82":{"cpu":"0.0","mem":"0.0"},"89":{"cpu":"101.6","mem":"0.4"},"98":{"cpu":"0.0","mem":"0.0"},"MiB":{"cpu":"4663.2","mem":"avail"},"PID":{"cpu":"%CPU","mem":"%MEM"},"Tasks:":{"cpu":"stopped,","mem":"0"},"top":{"cpu":"users,","mem":"load"}},"detection_fps":6.4,"detectors":{"coral_pci":{"detection_start":0.0,"inference_speed":11.88,"pid":621665}},"service":{"last_updated":1686555516,"latest_version":"0.12.0","storage":{"/dev/shm":{"free":2092.9,"mount_type":"tmpfs","total":2147.5,"used":54.6},"/media/frigate/clips":{"free":177344.5,"mount_type":"xfs","total":1098973.7,"used":921629.2},"/media/frigate/recordings":{"free":177344.5,"mount_type":"xfs","total":1098973.7,"used":921629.2},"/tmp/cache":{"free":945.0,"mount_type":"tmpfs","total":1000.0,"used":55.0}},"temperatures":{"apex_0":40.3},"uptime":313916,"version":"0.12.0-da3e197"}}

Operating system

Proxmox

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

Dahua

Any other information that may be helpful

Proxmox VM Config

8 VCPU 8 GB RAM Boot Disk 200G (SSD) Data Disk 1T (Spinning) PCI and USB (disabled in config) Coral passthrough

Location Used Total
/dev/shm 39.1 MB 2.10 GB
/tmp/cache 67.1 MB 1000 MB
Recordings & Snapshots 888.13 GB 1073.22 GB
-- -- --
NickM-27 commented 1 year ago

There are 2 problems here:

  1. You have connection issues: rtsp://127.0.0.1:8554/House-Front: Connection timed out you may want to update to a newer go2rtc

  2. There is not enough cache for that many cameras v_interleaved_write_frame(): No space left on device so it needs to be increased

Also, using the mp4 api endpoints (like in the ha blueprint, downloading mp4, etc) also uses cage so that could be part of the issue as well

ankycooper commented 1 year ago
  1. I'm running go2rtc 1.5.0 which the latest release.
  2. I'm using RAM-based cache as per my doc above it is only 67.1/1000MB used.

Don't download any MP4, the notification blueprint is only limited to 1 cam (outdoor-Room).

NickM-27 commented 1 year ago

Was the screenshot taken while this error was happening? Otherwise it is irrelevant. ffmpeg makes no mistake that it does not have space to write the mp4 so at that point it was indeed out of cache. To find out the reason why we would need to know what files were taking up space at that moment

skrashevich commented 1 year ago

how I can see, you process crashes after 23.5 hours. this is 84600 seconds. every 84600 seconds, by default, docker engine runs internal garbage collector....

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.