[Support]: High CPU Utilization and Motion Detection Too Sensitive

[greg520820]

Describe the problem you are having

After updating from 0.9.4 to 0.10 the cpu utilization increased significantly and the number of motion boxes appears to be the reasons. The increase in motion detection is happening on all four cameras. I'm using the default values for all motion parameters on both versions.

Looking for suggestions/settings to get the motion detection on version 0.10 closer to 0.9.4.

Motion boxes on version 0.9.4: Same view showing increased number of motion boxes on 0.10. It is very overcast today with no wind. When sunny there are 30 to 50 motion boxes in this view: On version 0.9.4 cpu utilization at 41%:

On version 0.10 cpu utilization at 102% and goes much higher on sunny day:

Version

0.10.0-BFECEE9

Frigate config file

mqtt:
  host: 192.168.0.217
  user: mqttadmin
  password: redacted

environment_vars:
  LIBVA_DRIVER_NAME: radeonsi

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - qsv
    - -qsv_device
    - /dev/dri/renderD128

cameras:
  side_yard:
    ffmpeg:
      inputs:
        - path: rtsp://admin:redacted@192.168.50.226:554
          roles:
            - detect
    detect:
      width: 3840
      height: 2160
      fps: 5
    rtmp:
      enabled: false
    objects:
      track:
        - car
        - person
      filters:
        car:
          mask:
            - 2640,23,2558,514,3587,699,3714,0 #clock upper drive
            - 3217,0,3239,305,2640,594,2403,733,1625,1061,0,1696,0,0 #trees
            - 3498,1192,3639,351,3528,319,3314,806,3041,840,2756,981,2574,972,2435,1181,2384,1308,2057,1618,1764,1625,1723,1919,1402,2160,2884,2160 #bushes
        person:
          mask:
            - 2925,0,2947,435,3635,526,3714,0 #clock upper drive
            #- 3136,0,3746,0,3680,341,2909,325 #clock
            - 3217,0,3239,305,2640,594,2403,733,1625,1061,0,1696,0,0 #trees
            - 3498,1192,3639,351,3528,319,3403,624,3089,685,2830,924,2497,1074,2384,1308,2057,1618,1764,1625,1717,1825,1402,2160,2884,2160 #bushes
    record:
      enabled: True
      retain_days: 0
      events:
        retain:
          default: 5
    snapshots:
      enabled: True
      timestamp: True
      crop: True
      retain:
        default: 14
    motion:
      #threshold: 25
      mask: 
        - 3136,0,3746,0,3680,341,2909,325 #clock
        - 3217,0,3239,305,2640,594,2403,733,1625,1061,0,1696,0,0 #trees
        - 3498,1192,3639,351,3528,319,3403,624,3089,685,2756,981,2526,954,2413,1020,2384,1308,2057,1618,1764,1625,1717,1825,1402,2160,2884,2160 #bushes

  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://admin:redacted@192.168.50.225:554
          roles:
            - detect
    detect:
      width: 3840
      height: 2160
      fps: 5
    rtmp:
      enabled: false
    objects:
      track:
        - person
        - car
      filters:
        person:
          # Optional: minimum score for the object to initiate tracking (default: shown below)
        #  min_score: 0.3
          # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
         # threshold: 0.5
          # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
          # Checks based on the bottom center of the bounding box of the object
          mask:
            - 2961,740,3057,1197,2595,1256,2458,1031,2713,733 #fountain
    record:
      enabled: True
      retain_days: 0
      events:
        retain:
          default: 5
    snapshots:
      enabled: True
      timestamp: True
      crop: True
      retain:
        default: 5
    motion:
      #threshold: 25
      mask:
        - 872,0,1425,107,3387,237,3840,296,3840,0,3138,0 #trees clock
        - 3840,353,3710,1236,3159,2160,2230,1773,2112,1368,2110,1029,2399,838,2672,737,2952,576,3246,503 #fountain area
        - 266,1072,587,1316,1548,744,1964,810,1577,1152,1511,1841,1033,2096,610,2160,0,1661,23,266 #left side of walk
        - 624,1172,717,981,615,906,203,933,378,1400
    zones:
      front_walk:
        coordinates: 995,2160,1331,1514,2335,1488,2668,2160 # 4K

  back_yard:
    ffmpeg:
      inputs:
        - path: rtsp://admin:redacted@192.168.50.227:554
          roles:
            - detect
    detect:   
       width: 3840
       height: 2160
       fps: 5
    rtmp:
      enabled: false
    objects:
      track:
        - person
      filters:
        person:
          mask:
            - 483,325,589,860,125,1163,0,801,0,489 # window
    record:
      enabled: True
      retain_days: 0
      events:
        retain:
          default: 5
    snapshots:
      enabled: True
      timestamp: True
      crop: True
      retain:
        default: 14
    motion:
      #threshold: 25  #for version .10
      mask:
        - 2810,365,3483,451,3638,547,3840,676,3840,0,3114,0,2857,0
        - 3840,1552,3840,2160,2727,2160,2749,1539
        - 483,325,589,860,125,1163,0,801,0,489 # window

  garage:
    ffmpeg:
      inputs:
        - path: rtsp://admin:redacted@192.168.50.228:554
          roles:
            - detect
    detect:    
      width: 3840
      height: 2160
      fps: 5
    rtmp:
      enabled: false
    record:
      enabled: True
      retain_days: 0
      events:
        retain:
          default: 5
    snapshots:
      enabled: True
      timestamp: True
      crop: True
      retain:
        default: 5
    motion:
      #threshold: 25
      mask:
        - 3840,774,1448,430,1486,0,3840,0
        - 469,314,528,523,503,621,191,751,93,380
        - 2540,976,3840,1473,3840,1311,2599,844
        #- 3762,2130,3769,1825,3007,1730,2444,1727,1691,1866,1163,2160 # bed by AC
        #- 3840,0,3840,2160,0,2160,0,0 # masks complete view
    zones:
      side_door:
        coordinates: 1193,2160,701,2160,464,1814,1058,1539,1557,1930   
detectors: 
  #coral_pci:
   # type: edgetpu
    #device: pci
 # coral1:
  #  type: edgetpu
   # device: pci:0
  cpu1:
    type: cpu
  cpu2:
    type: cpu
  cpu3:
    type: cpu
  cpu4:
    type: cpu
  cpu5:
    type: cpu 
birdseye:
  enabled: True
  width: 3840 #1280
  height: 2160 #720
  quality: 18

Relevant log output

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-02-22 15:58:59] frigate.app                    INFO    : Starting Frigate (0.10.0-bfecee9)
[2022-02-22 15:58:59] frigate.config                 WARNING : The 'retain_days' config option has been DEPRECATED and will be removed in a future version. Please use the 'days' setting under 'retain'
[2022-02-22 15:58:59] frigate.config                 WARNING : The 'retain_days' config option has been DEPRECATED and will be removed in a future version. Please use the 'days' setting under 'retain'
[2022-02-22 15:58:59] frigate.config                 WARNING : The 'retain_days' config option has been DEPRECATED and will be removed in a future version. Please use the 'days' setting under 'retain'
[2022-02-22 15:59:00] frigate.config                 WARNING : The 'retain_days' config option has been DEPRECATED and will be removed in a future version. Please use the 'days' setting under 'retain'
[2022-02-22 15:59:00] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2022-02-22 15:59:00] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-02-22 15:59:00] peewee_migrate                 INFO    : There is nothing to migrate
[2022-02-22 15:59:00] detector.cpu1                  INFO    : Starting detection process: 220
[2022-02-22 15:59:00] detector.cpu2                  INFO    : Starting detection process: 222
[2022-02-22 15:59:00] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2022-02-22 15:59:00] detector.cpu5                  INFO    : Starting detection process: 228
[2022-02-22 15:59:00] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2022-02-22 15:59:00] frigate.app                    INFO    : Output process started: 230
[2022-02-22 15:59:00] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2022-02-22 15:59:00] frigate.app                    INFO    : Camera processor started for side_yard: 232
[2022-02-22 15:59:00] detector.cpu4                  INFO    : Starting detection process: 226
[2022-02-22 15:59:00] detector.cpu3                  INFO    : Starting detection process: 224
[2022-02-22 15:59:00] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2022-02-22 15:59:00] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2022-02-22 15:59:00] frigate.app                    INFO    : Camera processor started for front_door: 235
[2022-02-22 15:59:00] frigate.app                    INFO    : Camera processor started for back_yard: 236
[2022-02-22 15:59:00] frigate.app                    INFO    : Camera processor started for garage: 238
[2022-02-22 15:59:00] frigate.app                    INFO    : Capture process started for side_yard: 239
[2022-02-22 15:59:00] frigate.app                    INFO    : Capture process started for front_door: 243
[2022-02-22 15:59:00] frigate.app                    INFO    : Capture process started for back_yard: 245
[2022-02-22 15:59:00] frigate.app                    INFO    : Capture process started for garage: 249
[2022-02-22 15:59:00] ws4py                          INFO    : Using epoll
[2022-02-22 15:59:00] ws4py                          INFO    : Using epoll
[2022-02-22 16:00:52] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:36232]
[2022-02-22 16:03:10] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:36232]
[2022-02-22 16:10:39] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:40648]
[2022-02-22 16:12:45] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:40648]

FFprobe output from your camera

NA

Frigate stats

No response

Operating system

Windows

Install method

HassOS Addon

Coral version

CPU (no coral)

Network connection

Wired

Camera make and model

Amcrest 4k ip8m-2496e

Any other information that may be helpful

No response

[NickM-27]

The release notes stated a change to make it less sensitive than 0.9.4 so becoming too sensitive would be inconsistent with that, not sure how that would be the case.

In any case, you can tune the motion sensitivity by adjusting

motion:
  contour_area:
  frame_height:

It will take some tuning but I would say a good starting place would be:

motion:
  contour_area: 50

if that doesn't work may need to also change frame_height to be lower.

[greg520820]

Yes, I read and followed the release notes, that is why I used the default settings for contour area, frame height, and motion threshold.

This following comment is not correct. The motion threshold default is 25 on version 0.10: (What about the motion threshold default setting. It appears it changed from 25 on version 0.9.4 to 5 on 0.10? Was this part of removing the dynamic motion sensitivity and is 5 without dynamic motion sensitivity, equal to 25 with dynamic motion sensitivity?)

I tried adjusting contour area and frame height and it didn't seem to effect motion sensitivity significantly. I did change the motion threshold back up to 25 and it did reduce the motion sensitivity and the number of motion boxes, but it seems that is not the parameter I'm suppose to be changing.

[blakeblackshear]

So a couple things changed related to motion detection:

0.9.4 would dynamically choose a frame_height and a contour_area based on the resolution of the camera. With high resolutions like this, that meant it was using a much larger frame for detecting motion. This would often result in much smaller details being detected as motion. 0.10.0 fixed those values because in reality the resolution shouldn't drive how fine the details are that count as motion. A car in a 720p frame takes up roughly the same % of the frame as a 4k camera. These changes would have reduced the sensitivity of motion detection for high resolutions like this.

In addition, 0.10.0 introduced dynamic contrast. Essentially it amplifies the differences between light and dark areas to improve motion detection under IR lights or in low contrast situations during the day. Those changes would have increased sensitivity of the motion detection.

So essentially, some changes increased sensitivity and others reduced it, but for very different reasons.

I believe in this case the increase in sensitivity is being driven by the dynamic contrast or the differences in the pixel values.

I would try increasing the threshold setting which will require a larger difference in the pixel value from one frame to the next to be considered motion. Increasing the contour_area will require that a larger area changed pixels will be near eachother. That will prevent small things like leaves from being detected as motion.

[blakeblackshear]

The default threshold is definitely still 25 in 0.10.0. What makes you think thats not the case?

[blakeblackshear]

Assuming the resolutions are the same, with 0.9.4 your previous motion settings would have been calculated as:

frame_height: 360
contour_area: 400

However, reverting to these values won't match the same behavior due to the new dynamic contrast.

I would recommend the following:

1. Try increasing your contour_area. If you find that small objects are not being detected as motion, reduce that number back to the default.
2. Try increasing your threshold. If you start missing motion, that means the difference in the pixel values isn't enough to cross the threshold.
3. You could also see if your cameras support Wide Dynamic Range (WDR) and enable that. It should increase the contrast on the camera dynamically which would reduce the amount Frigate dynamically adjusts contrast.
4. As a last resort, try increasing the frame_height and contour_area together while keeping the contour_area roughly the same % of the total pixels of the frame resolution.

Also, thanks a ton for the well crafted issue. Much appreciated.

[greg520820]

I thought I saw the motion threshold set at 5 in the config section of the debug page when version 0.10 was loaded, but I was wrong.

I will give your tuning adjustments a try. The cameras do support WDR, so I will also give that a try.

Thank you for the suggestions.

[TelepathicWalrus]

Did you have any luck in getting the right threshold and contour_area values? I've got a similar setup to you and need threshold=50 and contour_area=50 for motion detection to be reduced. Unfortunately this results in some motion being missed and still some false positives so I'm kind of stuck in the middle :P

[greg520820]

Tried Blake's suggestion of increasing contour_area first with little success. I started at 50 and slowly went up to 500. The only change was larger motion boxes, but at the same frequency and no improvement in cpu loading.

Next tried increasing the threshold and most of the motion boxes stopped in the 40 - 45 range, but motion was being missed and small object detection suffered, but it was probably due to lack of motion detection. Always had many false detections, i.e. rocks and fountains detected as people and cars.

Tried changing the contour_area and frame_height together and again could not significantly reduce the motion detection sensitivity.

Then I went back to the default settings for contour_area, frame_height, and threshold and tried greatly increasing the contrast settings on the cameras. I changed the contrast settings on the cameras from 55% to 90%. This stopped the overly sensitive motion detection, reduced cpu loading, and detection seemed better. The picture from the cameras doesn't look natural (see attached) and it seemed to make the false detections a little more frequent.

It appears the the dynamic contrast adjustment may be overly aggressive, at least for high resolution cameras. Maybe an adjustable gain parameter for the dynamic contrast would allow it to work for high resolution cameras?

At this point it was the end of February and my coral accelerator was suppose to ship on March 1. So, I went back to Frigate version 0.9.4 and planned to go back to version 0.10 with the coral. Of course it didn't ship, so I will probably stay on 0.9.4 until the coral ships or a new version of Frigate is released. I'm using the default 0.10 settings for contour_area, frame_height, and threshold on version 0.9.4 and they work fine.

Picture with contrast setting at 55%:

Picture with contrast setting at 90%:

[blakeblackshear]

It would be great if you could share a sample of a few minutes of video on Google Drive with blake@frigate.video. That way I can play with it myself. I may just make the dynamic contrast adjustable and/or only run it when cameras are in IR mode. To clarify, the contrast is adjusted dynamically only for motion. It does not impact the frame sent for object detection in any way. The false positives are just because object detection is running more frequently with the increased motion activity. More chances for a false positive.

[greg520820]

Sent the video. Yes, the increase in motion activation was 100X, so many more chances for false positives.

After increasing the camera's contrast settings the motion detections returned to normal and there was still an increase in false detections. I believe this was caused by the stark contrast of the picture/frame being used for detection.

[greg520820]

I have been able to restore version 0.9.4 from a backup many times while evaluating version 0.10. I have several version 0.9.4 backups, but they fail to revert to version 0.9.4. Any idea what is going on? The supervisor log is below:

22-03-10 23:10:54 INFO (MainThread) [supervisor.backups.manager] Found 10 backup files 22-03-10 23:10:54 INFO (MainThread) [supervisor.backups.manager] Found 10 backup files 22-03-10 23:11:13 INFO (MainThread) [supervisor.backups.manager] Partial-Restore b44c9ee5 start 22-03-10 23:11:17 INFO (MainThread) [supervisor.backups.manager] Restoring b44c9ee5 Docker config 22-03-10 23:11:17 INFO (MainThread) [supervisor.backups.manager] Restoring b44c9ee5 Repositories 22-03-10 23:11:18 WARNING (SyncWorker_1) [supervisor.addons.validate] Add-on have full device access, and selective device access in the configuration. Please report this to the maintainer of Frigate NVR (Full Access) Beta (0.11.0) 22-03-10 23:11:18 WARNING (SyncWorker_1) [supervisor.addons.validate] Add-on have full device access, and selective device access in the configuration. Please report this to the maintainer of Frigate NVR (Full Access) 22-03-10 23:11:18 INFO (MainThread) [supervisor.store] Loading add-ons from store: 67 all - 0 new - 0 remove 22-03-10 23:11:18 INFO (MainThread) [supervisor.backups.manager] Restoring b44c9ee5 Add-ons 22-03-10 23:11:24 INFO (MainThread) [supervisor.addons.addon] Restore config for addon ccab4aaf_frigate 22-03-10 23:11:24 ERROR (MainThread) [supervisor.jobs] Unhandled exception: 'ccab4aaf_frigate' Traceback (most recent call last): File "/usr/src/supervisor/supervisor/jobs/decorator.py", line 106, in wrapper return await self._method(*args, **kwargs) File "/usr/src/supervisor/supervisor/addons/init.py", line 367, in restore await addon.restore(tar_file) File "/usr/src/supervisor/supervisor/addons/addon.py", line 832, in restore restore_image = self._image(data[ATTR_SYSTEM]) File "/usr/src/supervisor/supervisor/addons/model.py", line 632, in _image return f"{config[ATTR_REPOSITORY]}/{self.arch}-addon-{config[ATTR_SLUG]}" File "/usr/src/supervisor/supervisor/addons/model.py", line 509, in arch if ATTR_IMAGE in self.data: File "/usr/src/supervisor/supervisor/addons/addon.py", line 157, in data return self.sys_addons.data.system[self.slug] KeyError: 'ccab4aaf_frigate' 22-03-10 23:11:24 WARNING (MainThread) [supervisor.backups.backup] Can't restore Add-on ccab4aaf_frigate: 22-03-10 23:11:24 INFO (MainThread) [supervisor.backups.manager] Partial-Restore b44c9ee5 done

[NickM-27]

It looks like at one point you may have switched between the normal and full-access modes and it doesn't like restoring a backup for the other (not 100% sure but that is what it seems.)

Frigate 0.10.1 was just released with the dynamic contrast disabled by default (and configurable) though so I would recommend maybe giving that one a go since that should solve the issue you are having here.

[greg520820]

I've been on "normal" for a long time. I've also restored version 0.9.4 from backups many times in the last two weeks and it has always worked until now. I'm not sure how HA backs up addons, but it seems it may only backup settings and pointers to github. Is it possible something on github has changed where it can't find version 0.9.4?

That's great news about version 10.1. I will give it a try and provide feedback.

[greg520820]

The problem I was having restoring the 0.9.4 backup was resolved by updating to Supervisor version 9.3. This is troubling since I hadn't made any changes to anything on HA except switching back and forth between Frigate versions. So the backup stopped working due to some change that was made external to my system. I've started making image backups.

Frigate version 10.1 certainly has solved the increased motion sensitivity issues that I was having. I have been able to use the motion parameters from version 0.9.4 with excellent results. I'm having some problems with stationary object detection, but will open another issue to address those.