[Config Support]: Help with recording storage

[yuckypants]

Describe the problem you are having

Recordings are enormous and are growing exponentially. In 10 minutes, it's grown 300MB.

Version

0.12.1-367D724

Frigate config file

mqtt:
  enabled: false

database:
  path: /db/frigate.db

detectors:
  coral:
    type: edgetpu
    device: usb

birdseye:
  enabled: True
  mode: continuous
  quality: 8

detect:
  enabled: True
  width: 1920
  height: 1080

objects:
  track:
    - person
    - dog
    - cat
    - car
    - motorcycle
    - bird

record:
  enabled: True
  retain:
    days: 0
    mode: motion
  events:
    retain:
      default: 3
      mode: active_objects
      objects:
        dog: 1
        cat: 1
        person: 3
        car: 3
        motorcycle: 3
        bird: 1

snapshots:
  enabled: true
  retain:
    default: 3
  clean_copy: false

cameras:
  family:
    ffmpeg:
      output_args: 
        record: preset-record-generic
      inputs:
        - path: rtsp://admin:mypw@192.168.50.71/cam/realmonitor?channel=1&subtype=0
          input_args: preset-rtsp-restream
          roles:
            - record
#        - path: rtsp://admin:mypw@192.168.50.71/cam/realmonitor?channel=1&subtype=1
#          roles:
            - detect
      hwaccel_args: preset-vaapi
  bedroom:
    ffmpeg:
      output_args: 
        record: preset-record-generic
      inputs:
        - path: rtsp://admin:mypw@192.168.50.154/cam/realmonitor?channel=1&subtype=0
          input_args: preset-rtsp-restream
          roles:
            - record
#        - path: rtsp://admin:mypw@192.168.50.154/cam/realmonitor?channel=1&subtype=1
#          roles:
            - detect
      hwaccel_args: preset-vaapi
  playroom:
    ffmpeg:
      output_args:
        record: preset-record-generic
      inputs:
        - path: rtsp://admin:mypw@192.168.50.34/cam/realmonitor?channel=1&subtype=0
          input_args: preset-rtsp-restream
          roles:
            - record
#        - path: rtsp://admin:mypw@192.168.50.34/cam/realmonitor?channel=1&subtype=1
#          roles:
            - detect
      hwaccel_args: preset-vaapi
  entry:
    ffmpeg:
      output_args:
        record: preset-record-generic
      inputs:
        - path: rtsp://admin:mypw@192.168.50.253/cam/realmonitor?channel=1&subtype=0
          input_args: preset-rtsp-restream
          roles:
            - record
#        - path: rtsp://admin:mypw@192.168.50.253/cam/realmonitor?channel=1&subtype=1
#          roles:
            - detect
      hwaccel_args: preset-vaapi

Relevant log output

2023-06-30 08:15:19.213485623  [INFO] Starting Frigate...
2023-06-30 08:15:20.684838696  [2023-06-30 08:15:20] frigate.app                    INFO    : Starting Frigate (0.12.1-367d724)
2023-06-30 08:15:20.743123591  [2023-06-30 08:15:20] peewee_migrate                 INFO    : Starting migrations
2023-06-30 08:15:20.750955461  [2023-06-30 08:15:20] peewee_migrate                 INFO    : There is nothing to migrate
2023-06-30 08:15:20.780141055  [2023-06-30 08:15:20] frigate.app                    INFO    : Output process started: 291
2023-06-30 08:15:20.789668932  [2023-06-30 08:15:20] frigate.app                    INFO    : Camera processor started for family: 297
2023-06-30 08:15:20.798313908  [2023-06-30 08:15:20] frigate.app                    INFO    : Camera processor started for bedroom: 298
2023-06-30 08:15:20.806864273  [2023-06-30 08:15:20] frigate.app                    INFO    : Camera processor started for playroom: 299
2023-06-30 08:15:20.816560674  [2023-06-30 08:15:20] frigate.app                    INFO    : Camera processor started for entry: 300
2023-06-30 08:15:20.828423460  [2023-06-30 08:15:20] frigate.app                    INFO    : Capture process started for family: 302
2023-06-30 08:15:20.840662028  [2023-06-30 08:15:20] frigate.app                    INFO    : Capture process started for bedroom: 306
2023-06-30 08:15:20.849475856  [2023-06-30 08:15:20] frigate.app                    INFO    : Capture process started for playroom: 310
2023-06-30 08:15:20.860206973  [2023-06-30 08:15:20] frigate.app                    INFO    : Capture process started for entry: 313
2023-06-30 08:15:23.419325196  [2023-06-30 08:15:20] detector.coral                 INFO    : Starting detection process: 289
2023-06-30 08:15:23.419332380  [2023-06-30 08:15:20] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2023-06-30 08:15:23.425004723  [2023-06-30 08:15:23] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2023-06-30 08:16:22.175688161  [2023-06-30 08:16:22] frigate.record                 WARNING : Unable to find file from recordings database: /media/frigate/recordings/2023-06-26/20/playroom/50.30.mp4

Frigate stats

{"bedroom":{"camera_fps":5.2,"capture_pid":306,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":312,"pid":298,"process_fps":5.2,"skipped_fps":0.0},"cpu_usages":{"%Cpu(s):":{"cpu":"id,","mem":"0.7"},"1":{"cpu":"0.0","mem":"0.0"},"108":{"cpu":"0.0","mem":"0.0"},"111":{"cpu":"31.0","mem":"3.1"},"118":{"cpu":"0.0","mem":"0.2"},"135":{"cpu":"0.3","mem":"0.1"},"136":{"cpu":"0.0","mem":"0.1"},"137":{"cpu":"0.0","mem":"0.1"},"138":{"cpu":"0.0","mem":"0.1"},"16":{"cpu":"0.0","mem":"0.0"},"17":{"cpu":"0.0","mem":"0.0"},"25":{"cpu":"0.0","mem":"0.0"},"26":{"cpu":"0.0","mem":"0.0"},"27":{"cpu":"0.0","mem":"0.0"},"28":{"cpu":"0.0","mem":"0.0"},"283":{"cpu":"0.0","mem":"1.0"},"288":{"cpu":"0.3","mem":"0.1"},"289":{"cpu":"0.0","mem":"1.4"},"29":{"cpu":"0.0","mem":"0.0"},"291":{"cpu":"2.0","mem":"1.3"},"297":{"cpu":"0.7","mem":"1.3"},"298":{"cpu":"0.7","mem":"1.3"},"299":{"cpu":"1.0","mem":"1.4"},"30":{"cpu":"0.0","mem":"0.0"},"300":{"cpu":"1.0","mem":"1.3"},"301":{"cpu":"0.0","mem":"0.2"},"302":{"cpu":"3.7","mem":"1.2"},"305":{"cpu":"0.0","mem":"0.7"},"306":{"cpu":"3.3","mem":"1.2"},"307":{"cpu":"75.0","mem":"1.0"},"31":{"cpu":"0.0","mem":"0.0"},"310":{"cpu":"4.0","mem":"1.2"},"311":{"cpu":"0.0","mem":"0.3"},"312":{"cpu":"70.3","mem":"1.0"},"313":{"cpu":"3.7","mem":"1.2"},"319":{"cpu":"0.0","mem":"0.2"},"32":{"cpu":"0.0","mem":"0.0"},"321":{"cpu":"72.3","mem":"1.0"},"323":{"cpu":"71.0","mem":"1.0"},"325":{"cpu":"0.0","mem":"0.6"},"33":{"cpu":"0.0","mem":"0.0"},"42":{"cpu":"0.0","mem":"0.0"},"43":{"cpu":"0.0","mem":"0.0"},"724":{"cpu":"0.0","mem":"0.0"},"80":{"cpu":"0.0","mem":"0.0"},"81":{"cpu":"0.0","mem":"0.0"},"83":{"cpu":"0.0","mem":"0.0"},"832":{"cpu":"0.3","mem":"0.0"},"834":{"cpu":"35.3","mem":"0.4"},"845":{"cpu":"0.0","mem":"0.0"},"90":{"cpu":"0.0","mem":"0.2"},"MiB":{"cpu":"5943.4","mem":"avail"},"PID":{"cpu":"%CPU","mem":"%MEM"},"Tasks:":{"cpu":"stopped,","mem":"0"},"top":{"cpu":"users,","mem":"load"}},"detection_fps":8.7,"detectors":{"coral":{"detection_start":0.0,"inference_speed":11.0,"pid":289}},"entry":{"camera_fps":5.0,"capture_pid":313,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":323,"pid":300,"process_fps":5.0,"skipped_fps":0.0},"family":{"camera_fps":5.0,"capture_pid":302,"detection_enabled":1,"detection_fps":4.6,"ffmpeg_pid":307,"pid":297,"process_fps":5.0,"skipped_fps":0.0},"gpu_usages":{"intel-vaapi":{"gpu":"7.06 %","mem":"- %"}},"playroom":{"camera_fps":4.9,"capture_pid":310,"detection_enabled":1,"detection_fps":4.1,"ffmpeg_pid":321,"pid":299,"process_fps":4.9,"skipped_fps":0.0},"service":{"last_updated":1688138369,"latest_version":"0.12.1","storage":{"/dev/shm":{"free":253.9,"mount_type":"tmpfs","total":268.4,"used":14.5},"/media/frigate/clips":{"free":1525625.6,"mount_type":"cifs","total":5939022.9,"used":4413397.2},"/media/frigate/recordings":{"free":1525625.9,"mount_type":"cifs","total":5939022.9,"used":4413397.0},"/tmp/cache":{"free":981.0,"mount_type":"tmpfs","total":1000.0,"used":19.0}},"temperatures":{},"uptime":249,"version":"0.12.1-367d724"}}

Operating system

Debian

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

I've used up almost 1TB in 4 days, so I've been manually deleting recordings to get them normalized. I'm also still using this as a testbed on internal cameras before I install my outside cams, so I appreciate that there's significantly more movement inside than there will be outside; however, this is still unsustainable.

What have I done wrong in the config?

[NickM-27]

Please show a screenshot of the storage page

[yuckypants]

Thanks

[NickM-27]

I don't think anything is wrong here. Based on the bitrates you have set on your cameras if the entire hour is kept then on average that is 1.45 GB per hour per camera or roughly 241 MB per 10 minutes per camera. So if your storage went up by 300MB in 10 minutes then that means a significant amount of recordings were not kept and the recordings are working correctly. If this is alarming then it sounds like you don't have enough storage or your cameras are set at too high of a bitrate.

[yuckypants]

Ok, thank you for reviewing. I will check the bitrate on the cameras themselves.

The NAS that I have this on also houses plex and backups - and it is alarming that I've lost so much space in so few days, so I think I need to reassess my storage situation.

I appreciate your feedback, your FAST replies, and your willingness to help.

[yuckypants]

Oh one more question, do I give up the ability to enable/disable detect, recordings, or snapshots with the way I have my configuration? I attempted to do it to only certain cameras on the GUI and it appears to do nothing - and I'm wondering if that's because I added the record: entry set in the config rather than per camera.

If so, is it safe to assume that if I add the entry under the specific camera variable that it will override the global, or should I outright remove the global in favor of the individual camera entry?

[NickM-27]

if you have a config that is the same for every camera there is no reason to apply it to each camera. There is on affect on the UI from that perspective, as long as the recordings button is enabled it can be used to turn off future recordings for that camera

[yuckypants]

Got it, thank you.