search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.16k stars 1.75k forks source link

Snapshots aren't being cleaned up #5603

Closed toddstar closed 1 year ago

toddstar commented 1 year ago

Describe the problem you are having

I had snapshots retain set between 0 & 2 for my 3 camera which has previously been working however got a space limit warning in HA, checked my frigate\clips folder and was nearly 14GB with files going back to 30/01/23.

I've dumped about half of them to clean up some needed space and then tested adding objects days (just has default originally) and adjusting default days but still have files over the limit. Also tried adding to global config rather than per camera but no files deleted from my clips folder. Screenshot 2023-02-27 181319

Version

0.12.0-27A31E7

Frigate config file

mqtt:
  host: 192.168.0.240
  user: "mqttuser"
  password: [ removed ]
environment_vars:
  LIBVA_DRIVER_NAME: i965
ffmpeg:
  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p

#ffmpeg:
#  hwaccel_args:
#    - -hwaccel
#    - qsv
#    - -c:v
#    - h264_qsv
#  output_args:
#    detect: -f rawvideo -vf hwdownload,format=nv12 -pix_fmt yuv420p

cameras:
  # Name of your camera
  Kitchen:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.0.18/live/ch00_1
          roles:
            - record
            #- restream
            - detect
    detect:
      enabled: True
      width: 1280
      height: 720
      fps: 5
    objects:
      #Optional: list of objects to track from labelmap.txt (default: shown below)
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2
        objects:
          person: 5
      height: 600
      quality: 100
      crop: true
    record:
      enabled: true
      retain:
        days: 0
      events:
        retain:
          default: 1
    mqtt:
      timestamp: False
      bounding_box: False
      crop: True
      height: 600
      quality: 100

  Living:
    ffmpeg:
      inputs:
        - path: rtsp://admin:L2523C22@192.168.0.108:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
          roles:
            - record
            #- restream
            - detect

    detect:
      enabled: True
      width: 2560
      height: 1440
      fps: 5
    objects:
      #Optional: list of objects to track from labelmap.txt (default: shown below)
      track:
        - person
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2
        objects:
          person: 5
      crop: true
      height: 600
      quality: 100
    record:
      enabled: true
      retain:
        days: 0
      events:
        retain:
          default: 0
    mqtt:
      timestamp: False
      bounding_box: False
      crop: True
      height: 600
      quality: 100

  office:
    ffmpeg:
      inputs:
        - path: rtsp://192.168.0.53:554/0/av0
          roles:
             - record
             #- restream
             - detect

    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      # Optional: list of objects to track from labelmap.txt (default: shown below)
      track:
        - person
        - car
        - motorcycle
        - bicycle
        - bus
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2
        objects:
          person: 5
      crop: true
      height: 600
      quality: 100

    record:
      enabled: true
      retain:
        days: 0
      events:
        retain:
          default: 2
    mqtt:
      timestamp: False
      bounding_box: False
      crop: True
      height: 600
      quality: 100

snapshots:
  retain:
    default: 2
    objects:
      person: 5

detectors:
  coral1:
    type: edgetpu
    device: pci:0
  coral2:
    type: edgetpu
    device: pci:1

Relevant log output

not sure how helpful general log will be but that's below (can turn on debug and add that later)

2023-02-27 18:14:18.885059808  [2023-02-27 18:14:18] frigate.app                    INFO    : Starting Frigate (0.12.0-27a31e7)
2023-02-27 18:14:18.915999387  [2023-02-27 18:14:18] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-02-27 18:14:18.918033123  [2023-02-27 18:14:18] peewee_migrate                 INFO    : Starting migrations
2023-02-27 18:14:18.923332656  [2023-02-27 18:14:18] peewee_migrate                 INFO    : There is nothing to migrate
2023-02-27 18:14:18.948003484  [2023-02-27 18:14:18] frigate.app                    INFO    : Output process started: 292
2023-02-27 18:14:19.062440280  [2023-02-27 18:14:18] frigate.app                    INFO    : Camera processor started for Kitchen: 300
2023-02-27 18:14:19.062443272  [2023-02-27 18:14:19] frigate.app                    INFO    : Camera processor started for Living: 309
2023-02-27 18:14:19.062447213  [2023-02-27 18:14:19] frigate.app                    INFO    : Camera processor started for office: 310
2023-02-27 18:14:19.064582526  [2023-02-27 18:14:19] frigate.app                    INFO    : Capture process started for Kitchen: 311
2023-02-27 18:14:19.067912637  [2023-02-27 18:14:19] frigate.app                    INFO    : Capture process started for Living: 312
2023-02-27 18:14:19.073186595  [2023-02-27 18:14:19] frigate.app                    INFO    : Capture process started for office: 313
2023-02-27 18:14:19.196805395  [2023-02-27 18:14:18] detector.coral2                INFO    : Starting detection process: 291
2023-02-27 18:14:19.196985942  [2023-02-27 18:14:18] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci:1
2023-02-27 18:14:19.199597228  [2023-02-27 18:14:18] detector.coral1                INFO    : Starting detection process: 290
2023-02-27 18:14:19.199600065  [2023-02-27 18:14:18] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci:0
2023-02-27 18:14:19.202114093  [2023-02-27 18:14:19] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2023-02-27 18:14:19.203988842  [2023-02-27 18:14:19] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2023-02-27 18:14:49.105274726  [2023-02-27 18:14:49] watchdog.office                INFO    : No frames received from office in 20 seconds. Exiting ffmpeg...
2023-02-27 18:14:49.106266363  [2023-02-27 18:14:49] watchdog.office                INFO    : Waiting for ffmpeg to exit gracefully...
2023-02-27 18:15:19.128135666  [2023-02-27 18:15:19] watchdog.office                INFO    : FFmpeg did not exit. Force killing...
2023-02-27 18:15:19.147370228  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.148327007  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.148577294  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.149196491  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.150699619  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.151218003  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.151220733  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.151222075  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.151223342  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.151234691  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.151236472  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.152018993  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: Unable to read frames from ffmpeg process.
2023-02-27 18:15:19.152167810  [2023-02-27 18:15:19] frigate.video                  ERROR   : office: ffmpeg process is not running. exiting capture thread...
2023-02-27 18:15:19.405697123  [2023-02-27 18:15:19] frigate.record                 WARNING : Failed to probe corrupt segment office-20230227181421.mp4: 0 - b'[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55d3ab4c66c0] moov atom not found\n/tmp/cache/office-20230227181421.mp4: Invalid data found when processing input\n'
2023-02-27 18:15:19.405806616  [2023-02-27 18:15:19] frigate.record                 WARNING : Discarding a corrupt recording segment: office-20230227181421.mp4
2023-02-27 18:15:29.154782852  [2023-02-27 18:15:29] watchdog.office                ERROR   : Ffmpeg process crashed unexpectedly for office.
2023-02-27 18:15:29.155300340  [2023-02-27 18:15:29] watchdog.office                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-02-27 18:15:29.155305100  [2023-02-27 18:15:29] ffmpeg.office.detect           ERROR   : [segment @ 0x55ece6f34400] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly

FFprobe output from your camera

N/A

Frigate stats

No response

Operating system

HassOS

Install method

HassOS Addon

Coral version

M.2

Network connection

Wired

Camera make and model

n/a

Any other information that may be helpful

No response

NickM-27 commented 1 year ago

Have you deleted the DB at any point? Frigate will not clean up images which it does not know about (meaning ones that are not stored in the DB). I'd say it is most likely this since no one else has reported an issue with this and the snapshot retention was not touched in 0.12

I'd suggest enabling debug logs for the events module and let it run for a bit, paste the logs here

toddstar commented 1 year ago

Have you deleted the DB at any point? Frigate will not clean up images which it does not know about (meaning ones that are not stored in the DB). I'd say it is most likely this since no one else has reported an issue with this and the snapshot retention was not touched in 0.12

I'd suggest enabling debug logs for the events module and let it run for a bit, paste the logs here

No haven't manually deleted the db.

Is it frigate.event for debug logger?

NickM-27 commented 1 year ago 
logger:
  logs:
    frigate.events: debug
toddstar commented 1 year ago

been through the clips files and all the overage images are from one camera (living) so seems to be connected with event retain set to 0, which with some further testing I've caught below at 20:42 but then has seeming worked correctly at 21:02

20:42

2023-02-27 20:32:19.034783111  [2023-02-27 20:32:19] frigate.app                    INFO    : Starting Frigate (0.12.0-27a31e7)
2023-02-27 20:32:19.067201876  [2023-02-27 20:32:19] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-02-27 20:32:19.067299784  [2023-02-27 20:32:19] peewee_migrate                 INFO    : Starting migrations
2023-02-27 20:32:19.072110824  [2023-02-27 20:32:19] peewee_migrate                 INFO    : There is nothing to migrate
2023-02-27 20:32:19.106117963  [2023-02-27 20:32:19] frigate.app                    INFO    : Output process started: 298
2023-02-27 20:32:19.106803527  [2023-02-27 20:32:19] detector.coral2                INFO    : Starting detection process: 296
2023-02-27 20:32:19.112641018  [2023-02-27 20:32:19] frigate.app                    INFO    : Camera processor started for Kitchen: 302
2023-02-27 20:32:19.117287951  [2023-02-27 20:32:19] frigate.app                    INFO    : Camera processor started for Living: 304
2023-02-27 20:32:19.120855501  [2023-02-27 20:32:19] frigate.app                    INFO    : Camera processor started for office: 305
2023-02-27 20:32:19.124651050  [2023-02-27 20:32:19] frigate.app                    INFO    : Capture process started for Kitchen: 306  
2023-02-27 20:32:19.129324855  [2023-02-27 20:32:19] frigate.app                    INFO    : Capture process started for Living: 307
2023-02-27 20:32:19.134671670  [2023-02-27 20:32:19] frigate.app                    INFO    : Capture process started for office: 308
2023-02-27 20:32:19.361763984  [2023-02-27 20:32:19] detector.coral1                INFO    : Starting detection process: 291
2023-02-27 20:32:19.361768386  [2023-02-27 20:32:19] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci:0
2023-02-27 20:32:19.363559322  [2023-02-27 20:32:19] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci:1
2023-02-27 20:32:19.368749955  [2023-02-27 20:32:19] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2023-02-27 20:32:19.369161268  [2023-02-27 20:32:19] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2023-02-27 20:32:21.627034529  [2023-02-27 20:32:21] frigate.events                 DEBUG   : Event received: start office 1677529939.178958-s34io5
2023-02-27 20:32:21.773129217  [2023-02-27 20:32:21] frigate.events                 DEBUG   : Event received: start office 1677529941.69626-szz7qx
2023-02-27 20:32:21.773202505  [2023-02-27 20:32:21] frigate.events                 DEBUG   : Event received: start office 1677529941.69626-4gy18k
2023-02-27 20:32:31.325459016  [2023-02-27 20:32:31] frigate.events                 DEBUG   : Event received: end office 1677529941.69626-szz7qx
2023-02-27 20:32:32.365410744  [2023-02-27 20:32:32] frigate.events                 DEBUG   : Event received: end office 1677529939.178958-s34io5
2023-02-27 20:37:13.560261836  [2023-02-27 20:37:13] frigate.events                 DEBUG   : Event received: start office 1677530233.376443-iiwieu
2023-02-27 20:37:13.950156208  [2023-02-27 20:37:13] frigate.events                 DEBUG   : Event received: start office 1677530233.777248-xapj3v
2023-02-27 20:37:16.202569160  [2023-02-27 20:37:16] frigate.events                 DEBUG   : Event received: start office 1677530236.017139-osgwvj
2023-02-27 20:37:22.606096287  [2023-02-27 20:37:22] frigate.events                 DEBUG   : Event received: end office 1677530233.376443-iiwieu
2023-02-27 20:37:23.256337634  [2023-02-27 20:37:23] frigate.events                 DEBUG   : Event received: start Kitchen 1677530243.042117-5pwxtw
2023-02-27 20:37:24.526868670  [2023-02-27 20:37:24] frigate.events                 DEBUG   : Event received: update Kitchen 1677530243.042117-5pwxtw
2023-02-27 20:37:24.631838564  [2023-02-27 20:37:24] frigate.events                 DEBUG   : Event received: end office 1677530236.017139-osgwvj
2023-02-27 20:37:25.105632200  [2023-02-27 20:37:25] frigate.events                 DEBUG   : Event received: start Kitchen 1677530244.857009-4thqkb
2023-02-27 20:37:30.309037066  [2023-02-27 20:37:30] frigate.events                 DEBUG   : Event received: end Kitchen 1677530244.857009-4thqkb
2023-02-27 20:37:32.917953327  [2023-02-27 20:37:32] frigate.events                 DEBUG   : Event received: start Living 1677530252.632851-kbuf2k
2023-02-27 20:37:33.269558101  [2023-02-27 20:37:33] frigate.events                 DEBUG   : Event received: update Living 1677530252.632851-kbuf2k
2023-02-27 20:37:35.300170623  [2023-02-27 20:37:35] frigate.events                 DEBUG   : Event received: start Living 1677530255.053213-fkibvl
2023-02-27 20:37:37.489263548  [2023-02-27 20:37:37] frigate.events                 DEBUG   : Event received: update Living 1677530255.053213-fkibvl
2023-02-27 20:37:38.286294841  [2023-02-27 20:37:38] frigate.events                 DEBUG   : Event received: update Living 1677530252.632851-kbuf2k
2023-02-27 20:37:38.956657329  [2023-02-27 20:37:38] frigate.events                 DEBUG   : Event received: start Living 1677530258.667973-8jv9p6
2023-02-27 20:37:39.159761699  [2023-02-27 20:37:39] frigate.events                 DEBUG   : Event received: end Kitchen 1677530243.042117-5pwxtw
2023-02-27 20:37:42.695207750  [2023-02-27 20:37:42] frigate.events                 DEBUG   : Event received: update Living 1677530255.053213-fkibvl
2023-02-27 20:37:45.252829520  [2023-02-27 20:37:45] frigate.events                 DEBUG   : Event received: end Living 1677530258.667973-8jv9p6
2023-02-27 20:37:47.679700085  [2023-02-27 20:37:47] frigate.events                 DEBUG   : Event received: update Living 1677530255.053213-fkibvl
2023-02-27 20:37:49.863199539  [2023-02-27 20:37:49] frigate.events                 DEBUG   : Event received: update Living 1677530252.632851-kbuf2k
2023-02-27 20:37:52.874992982  [2023-02-27 20:37:52] frigate.events                 DEBUG   : Event received: update Living 1677530255.053213-fkibvl
2023-02-27 20:37:59.899791933  [2023-02-27 20:37:59] frigate.events                 DEBUG   : Event received: update Living 1677530252.632851-kbuf2k
2023-02-27 20:37:59.913804562  [2023-02-27 20:37:59] frigate.events                 DEBUG   : Event received: update Living 1677530255.053213-fkibvl
2023-02-27 20:38:04.876642451  [2023-02-27 20:38:04] frigate.events                 DEBUG   : Event received: update Living 1677530255.053213-fkibvl
2023-02-27 20:38:10.095240283  [2023-02-27 20:38:10] frigate.events                 DEBUG   : Event received: update Living 1677530255.053213-fkibvl
2023-02-27 20:38:33.860324589  [2023-02-27 20:38:33] frigate.events                 DEBUG   : Event received: update Living 1677530255.053213-fkibvl
2023-02-27 20:38:38.457384484  [2023-02-27 20:38:38] frigate.events                 DEBUG   : Event received: update Living 1677530252.632851-kbuf2k
2023-02-27 20:38:41.261805190  [2023-02-27 20:38:41] frigate.events                 DEBUG   : Event received: update Living 1677530255.053213-fkibvl
2023-02-27 20:38:41.690990175  [2023-02-27 20:38:41] frigate.events                 DEBUG   : Event received: update Living 1677530252.632851-kbuf2k
2023-02-27 20:38:50.033283631  [2023-02-27 20:38:50] frigate.events                 DEBUG   : Event received: end Living 1677530255.053213-fkibvl
2023-02-27 20:38:50.962554806  [2023-02-27 20:38:50] frigate.events                 DEBUG   : Event received: start Kitchen 1677530329.513221-0c63m5
2023-02-27 20:38:51.647097618  [2023-02-27 20:38:51] frigate.events                 DEBUG   : Event received: end Living 1677530252.632851-kbuf2k
2023-02-27 20:38:54.895667167  [2023-02-27 20:38:54] frigate.events                 DEBUG   : Event received: end Kitchen 1677530329.513221-0c63m5
2023-02-27 20:41:54.433516469  [2023-02-27 20:41:54] frigate.events                 DEBUG   : Event received: start Living 1677530514.238383-loz2nm
2023-02-27 20:41:54.661084211  [2023-02-27 20:41:54] frigate.events                 DEBUG   : Event received: update Living 1677530514.238383-loz2nm
2023-02-27 20:41:58.455435670  [2023-02-27 20:41:58] frigate.events                 DEBUG   : Event received: start Living 1677530518.263472-c4jt3h
2023-02-27 20:41:59.663638934  [2023-02-27 20:41:59] frigate.events                 DEBUG   : Event received: update Living 1677530514.238383-loz2nm
2023-02-27 20:41:59.681318837  [2023-02-27 20:41:59] frigate.events                 DEBUG   : Event received: update Living 1677530518.263472-c4jt3h
2023-02-27 20:42:00.708091420  [2023-02-27 20:42:00] frigate.events                 DEBUG   : Event received: start Living 1677530520.490743-gycc7h
2023-02-27 20:42:04.735299580  [2023-02-27 20:42:04] frigate.events                 DEBUG   : Event received: update Living 1677530518.263472-c4jt3h
2023-02-27 20:42:05.869253129  [2023-02-27 20:42:05] frigate.events                 DEBUG   : Event received: end Living 1677530520.490743-gycc7h
2023-02-27 20:42:06.425343719  [2023-02-27 20:42:06] frigate.events                 DEBUG   : Event received: end Living 1677530514.238383-loz2nm
2023-02-27 20:42:09.915603702  [2023-02-27 20:42:09] frigate.events                 DEBUG   : Event received: update Living 1677530518.263472-c4jt3h
2023-02-27 20:42:13.676112273  [2023-02-27 20:42:13] frigate.events                 DEBUG   : Event received: start Living 1677530533.454108-a7wly9
2023-02-27 20:42:17.773885045  [2023-02-27 20:42:17] frigate.events                 DEBUG   : Event received: start Kitchen 1677530537.592598-p5gbjk
2023-02-27 20:42:18.025805316  [2023-02-27 20:42:18] frigate.events                 DEBUG   : Event received: update Kitchen 1677530537.592598-p5gbjk
2023-02-27 20:42:19.684064955  [2023-02-27 20:42:19] frigate.events                 DEBUG   : Removing duplicate: 1677530255.053213-fkibvl
2023-02-27 20:42:19.834533706  [2023-02-27 20:42:19] frigate.events                 DEBUG   : Event received: start Kitchen 1677530539.606346-uxh3x7
2023-02-27 20:42:20.678629453  [2023-02-27 20:42:20] frigate.events                 DEBUG   : Event received: end Living 1677530518.263472-c4jt3h
2023-02-27 20:42:22.677658936  [2023-02-27 20:42:22] frigate.events                 DEBUG   : Event received: end Living 1677530533.454108-a7wly9
2023-02-27 20:42:23.020372961  [2023-02-27 20:42:23] frigate.events                 DEBUG   : Event received: update Kitchen 1677530537.592598-p5gbjk
2023-02-27 20:42:25.503104371  [2023-02-27 20:42:25] frigate.events                 DEBUG   : Event received: end Kitchen 1677530539.606346-uxh3x7
2023-02-27 20:42:28.280089326  [2023-02-27 20:42:28] frigate.events                 DEBUG   : Event received: update Kitchen 1677530537.592598-p5gbjk
2023-02-27 20:42:34.921021391  [2023-02-27 20:42:34] frigate.events                 DEBUG   : Event received: end Kitchen 1677530537.592598-p5gbjk
2023-02-27 20:42:38.351544976  [2023-02-27 20:42:38] frigate.events                 DEBUG   : Event received: update office 1677529941.69626-4gy18k
2023-02-27 20:42:39.393056960  [2023-02-27 20:42:39] frigate.events                 DEBUG   : Event received: start office 1677530559.215432-iajbke
2023-02-27 20:42:43.550543255  [2023-02-27 20:42:43] frigate.events                 DEBUG   : Event received: update office 1677529941.69626-4gy18k
2023-02-27 20:42:45.140599472  [2023-02-27 20:42:45] frigate.events                 DEBUG   : Event received: end office 1677530233.777248-xapj3v
2023-02-27 20:42:45.722560867  [2023-02-27 20:42:45] frigate.events                 DEBUG   : Event received: end office 1677529941.69626-4gy18k

event has ended and been cleared from events i can see front end but files left in clips Screenshot 2023-02-27 211445

21:02

2023-02-27 21:02:05.440652885  [2023-02-27 21:02:05] frigate.events                 DEBUG   : Event received: start office 1677531725.174257-tfqtpb
2023-02-27 21:02:12.146778011  [2023-02-27 21:02:12] frigate.events                 DEBUG   : Event received: end office 1677531725.174257-tfqtpb
2023-02-27 21:02:16.194214609  [2023-02-27 21:02:16] frigate.events                 DEBUG   : Event received: start Kitchen 1677531735.990446-h9bwkc
2023-02-27 21:02:16.359277054  [2023-02-27 21:02:16] frigate.events                 DEBUG   : Event received: update Kitchen 1677531735.990446-h9bwkc
2023-02-27 21:02:16.593808423  [2023-02-27 21:02:16] frigate.events                 DEBUG   : Event received: start Kitchen 1677531736.339096-5cq45c
2023-02-27 21:02:21.559329586  [2023-02-27 21:02:21] frigate.events                 DEBUG   : Event received: update Kitchen 1677531735.990446-h9bwkc
2023-02-27 21:02:21.924787042  [2023-02-27 21:02:21] frigate.events                 DEBUG   : Event received: end Kitchen 1677531736.339096-5cq45c
2023-02-27 21:02:27.274073371  [2023-02-27 21:02:27] frigate.events                 DEBUG   : Event received: start Living 1677531747.055344-xivnvm
2023-02-27 21:02:27.669331754  [2023-02-27 21:02:27] frigate.events                 DEBUG   : Event received: update Living 1677531747.055344-xivnvm
2023-02-27 21:02:28.685947427  [2023-02-27 21:02:28] frigate.events                 DEBUG   : Event received: start Living 1677531748.502981-0ky61z
2023-02-27 21:02:32.864245456  [2023-02-27 21:02:32] frigate.events                 DEBUG   : Event received: update Living 1677531747.055344-xivnvm
2023-02-27 21:02:33.911230995  [2023-02-27 21:02:33] frigate.events                 DEBUG   : Event received: end Kitchen 1677531735.990446-h9bwkc
2023-02-27 21:02:34.706111487  [2023-02-27 21:02:34] frigate.events                 DEBUG   : Event received: end Living 1677531748.502981-0ky61z
2023-02-27 21:02:36.894343721  [2023-02-27 21:02:36] frigate.events                 DEBUG   : Event received: start Living 1677531756.646937-j7z737
2023-02-27 21:02:37.905866188  [2023-02-27 21:02:37] frigate.events                 DEBUG   : Event received: update Living 1677531747.055344-xivnvm
2023-02-27 21:02:42.072347679  [2023-02-27 21:02:42] frigate.events                 DEBUG   : Event received: end Living 1677531756.646937-j7z737
2023-02-27 21:02:43.083771327  [2023-02-27 21:02:43] frigate.events                 DEBUG   : Event received: update Living 1677531747.055344-xivnvm
2023-02-27 21:02:44.302579550  [2023-02-27 21:02:44] frigate.events                 DEBUG   : Event received: start Kitchen 1677531764.111029-y2uyro
2023-02-27 21:02:44.536411602  [2023-02-27 21:02:44] frigate.events                 DEBUG   : Event received: update Kitchen 1677531764.111029-y2uyro
2023-02-27 21:02:49.487199287  [2023-02-27 21:02:49] frigate.events                 DEBUG   : Event received: start Kitchen 1677531769.32004-3qaa0v
2023-02-27 21:02:49.494190172  [2023-02-27 21:02:49] frigate.events                 DEBUG   : Event received: update Kitchen 1677531764.111029-y2uyro
2023-02-27 21:02:51.114025925  [2023-02-27 21:02:51] frigate.events                 DEBUG   : Event received: end Living 1677531747.055344-xivnvm
2023-02-27 21:02:57.323805877  [2023-02-27 21:02:57] frigate.events                 DEBUG   : Event received: end Kitchen 1677531769.32004-3qaa0v
2023-02-27 21:02:59.036483635  [2023-02-27 21:02:59] frigate.events                 DEBUG   : Event received: end Kitchen 1677531764.111029-y2uyro
2023-02-27 21:03:01.186397368  [2023-02-27 21:03:01] frigate.events                 DEBUG   : Event received: start office 1677531780.934274-djw89i
2023-02-27 21:03:06.544642555  [2023-02-27 21:03:06] frigate.events                 DEBUG   : Event received: end office 1677531780.934274-djw89i

clips where created for living & kitchen camera at 21:02 event Screenshot 2023-02-27 210328 but this time has deleted the images in the clips folder after the event ended Screenshot 2023-02-27 211742

NickM-27 commented 1 year ago

These are just showing events that have come through as new. I don't see any of the cleanup which seems to be correct, I had incorrectly assumed there were logs for the event cleanup process https://github.com/blakeblackshear/frigate/blob/a554b22968f524cd883fdec7ae6dbde4b5a2f2c7/frigate/events.py#L291

After the event ends we absolutely expect the images to remain there until the number of days has passed.

toddstar commented 1 year ago

After the event ends we absolutely expect the images to remain there until the number of days has passed.

which seems to be (somewhat) working if event retain is more then 0 (not sure why kitchen got deleted in above screenshot. Will test again after updating my config file)

    record:
      enabled: true
      retain:
        days: 0
      events:
        retain:
          default: 1

but if event retain is 0

    record:
      enabled: true
      retain:
        days: 0
      events:
        retain:
          default: 0

like I have for my living camera then I'd guess the event is sometimes being deleted before the clip image has been deleted so gets permanently left behind as the event is no longer in the db so image files can't be deleted?

NickM-27 commented 1 year ago

Those settings are for recordings which are entirely separate from snapshots (which is all that is kept in the clips directory).

Also if you're going to set 0 for both you should just disable recordings instead.

toddstar commented 1 year ago

Those settings are for recordings which are entirely separate from snapshots (which is all that is kept in the clips directory).

but with (recording and) events set to 0, (i believe) it doesn't write any data to the DB so the clips clean up doesn't work, as there is no link / lines in the db for the clean up to use.

Also if you're going to set 0 for both you should just disable recordings instead.

For the camera in question all I'm interested in is facial recognition via double take & motion/person detection for lights & other automations which i believe needs both roles? (recording when unoccupied and whatnot is done on synology surveillance station). Can change it to 1 day to avoid the clips folder expanding every day but will create a fair amount of unneeded recordings.

NickM-27 commented 1 year ago

Those settings are for recordings which are entirely separate from snapshots (which is all that is kept in the clips directory).

but with (recording and) events set to 0, (i believe) it doesn't write any data to the DB so the clips clean up doesn't work, as there is no link / lines in the db for the clean up to use.

That is incorrect, both the record and event configs only apply to recordings, snapshots has its own config and own DB entries.

Also if you're going to set 0 for both you should just disable recordings instead.

For the camera in question all I'm interested in is facial recognition via double take & motion/person detection for lights & other automations which i believe needs both roles? (recording when unoccupied and whatnot is done on synology surveillance station). Can change it to 1 day to avoid the clips folder expanding every day but will create a fair amount of unneeded recordings.

No that is incorrect, you should simply disable recordings since they are not being used

cameras:
  the_camera:
    ffmpeg:
      ...
    record:
      enabled: false
toddstar commented 1 year ago

where should the DB entries be? i don't have any listings for my living room camera in either events table or the recordings table

NickM-27 commented 1 year ago

it would be the events table, which is what handles the snapshots.

toddstar commented 1 year ago

No idea what the issue is/was but after renaming my old.frigate.zip db and restarting frigate its now behaving correctly with the new frigate.zip db it created, but I can't see any differences in the tables, columns or the database settings between the two.

It is definitely behaving differently though as frigate frontend events list never showed anything for my living room camera previously, where as it now lists events for ~5 mins before being deleted when days is set to 0

NickM-27 commented 1 year ago

Interesting, that is odd, makes sense though now that it is working as expected. At this point I think this can be closed, feel free to create a new issue if something else comes up.