Reduce object detection rate during storms

[silentnyte]

Describe what you are trying to accomplish and why in non technical terms It would be great if during high motion events (like snow or rain storms) the object detection rate would reduce. I have noticed that during snow and rain storms my NVR machine goes into overdrive. I do not see any value in processing these images because they will mostly end up mis-identifications anyways.

Describe the solution you'd like Set a threshold of percent of image that detects motion for x amount of time then reduce object detection rate to y. Might also be good to be able to increase the object percent confidence threshold.

[NickM-27]

This can already be done somewhat using the motion values inputs in MQTT to do things like reducing the motion sensitivity.

Or, if like you said you don't see any value in processing the images then can just disable object detection until the rain has stopped?

[silentnyte]

Are you suggesting using a external program to monitor the MQTT motion value then disable or reduce object detection? A solution within Frigate would be more robust and not rely on an external program to return to normal operation. Within Frigate a more robust detection of snow or rain storm could also be developed.

[NickM-27]

Are you suggesting using a external program to monitor the MQTT motion value then disable or reduce object detection? A solution within Frigate would be more robust and not rely on an external program to return to normal operation. Within Frigate a more robust detection of snow or rain storm could also be developed.

Home Assistant and other applications use MQTT to automate things, that way you can be very specific with how you want the situation to be tuned and handled. VS a more generic solution that would be implemented in frigate, which is why I suggested that.

[silentnyte]

I do not see in the setting where you can determine the % motion detected via MQTT. The only option I see would be to detected the time motion has been active. This would not be a good indicator of a snow or rain storm.

Is there another indicator of % area of motion detected?

[NickM-27]

Is there another indicator of % area of motion detected?

You could do it based on the current detection FPS. If the current detection fps is high for a long enough period of time then can have an action to lower motion sensitivity.

% motion detected will have its own problems, for example a garbage truck driving past the camera slowly as it picks up trash. It would need to be done very carefully to avoid false positive rain / snow detections which will probably vary for each persons camera.

For my setup, it makes a lot more sense to use other signals besides the camera to detect when it is raining / snowing like my weather station,

[silentnyte]

That is why I suggested a % area detected along with time. This would reduce the false positives.

How do you get the current detection FPS via MQTT?

[NickM-27]

If you use home assistant it is a diagnostic sensor, likely disabled by default. Otherwise, it is in the /stats endpoint of MQTT

[silentnyte]

How do you enable the additional diagnostics? Are there instructions for understanding frigate/stats, frigate//motion_threshold and frigate//motion_contour_area?

[NickM-27]

How do you enable the additional diagnostics?

Just click on the +x entities not shown part and then click on the one you want and press enable

Are there instructions for understanding frigate/stats, frigate//motion_threshold and frigate//motion_contour_area?

yes, they are explained in the reference config https://docs.frigate.video/configuration/index

[NickM-27]

# Optional: Motion configuration
# NOTE: Can be overridden at the camera level
motion:
  # Optional: The threshold passed to cv2.threshold to determine if a pixel is different enough to be counted as motion. (default: shown below)
  # Increasing this value will make motion detection less sensitive and decreasing it will make motion detection more sensitive.
  # The value should be between 1 and 255.
  threshold: 25
  # Optional: Minimum size in pixels in the resized motion image that counts as motion (default: 30)
  # Increasing this value will prevent smaller areas of motion from being detected. Decreasing will
  # make motion detection more sensitive to smaller moving objects.
  # As a rule of thumb:
  #  - 15 - high sensitivity
  #  - 30 - medium sensitivity
  #  - 50 - low sensitivity
  contour_area: 30
  # Optional: Alpha value passed to cv2.accumulateWeighted when averaging the motion delta across multiple frames (default: shown below)
  # Higher values mean the current frame impacts the delta a lot, and a single raindrop may register as motion.
  # Too low and a fast moving person wont be detected as motion.
  delta_alpha: 0.2
  # Optional: Alpha value passed to cv2.accumulateWeighted when averaging frames to determine the background (default: shown below)
  # Higher values mean the current frame impacts the average a lot, and a new object will be averaged into the background faster.
  # Low values will cause things like moving shadows to be detected as motion for longer.
  # https://www.geeksforgeeks.org/background-subtraction-in-an-image-using-concept-of-running-average/
  frame_alpha: 0.2
  # Optional: Height of the resized motion frame  (default: 50)
  # This operates as an efficient blur alternative. Higher values will result in more granular motion detection at the expense
  # of higher CPU usage. Lower values result in less CPU, but small changes may not register as motion.
  frame_height: 50
  # Optional: motion mask
  # NOTE: see docs for more detailed info on creating masks
  mask: 0,900,1080,900,1080,1920,0,1920
  # Optional: improve contrast (default: shown below)
  # Enables dynamic contrast improvement. This should help improve night detections at the cost of making motion detection more sensitive
  # for daytime.
  improve_contrast: False
  # Optional: Delay when updating camera motion through MQTT from ON -> OFF (default: shown below).
  mqtt_off_delay: 30

[silentnyte]

I do not have the + button. I am interested in add the "ffmpeg cpu usage"

[NickM-27]

I do not have the + button. I am interested in add the "ffmpeg cpu usage"

oh I see, that is only available in frigate 0.12 + frigate integration 4.0 (both currently in beta)

[silentnyte]

Gotcha, I will wait for it get out of beta then upgrade. I want to thank you for all the advice and help. I am going to try and creating a blueprint for this in HA.

I would like to leave this open as a future feature request. This would be much more robust if developed within Frigate.

I also wanted to share why I am requesting this feature. Here is a snapshot of my NVR CPU load and temp while it is snowing.

[blakeblackshear]

Are you using a Coral or one of the new detectors available in 0.12? Or is this CPU usage just because you are using CPU detectors?

[silentnyte]

I am not able to get my hands on a TPU for a reasonable price, so I am just using CPU detection.

I logged the MQTT messages and there is no info that would be useful in determining that there is excessive motion activity causing high CPU usage. I see that there might be more info in v12 beta. I may update to see if I can come up with something.

Do you see this as something that could be developed internal to Frigate in the future?

[NickM-27]

I logged the MQTT messages and there is no info that would be useful in determining that there is excessive motion activity causing high CPU usage.

Detect_fps is there and that's what you'd want to key on

[silentnyte]

It changes from 7 to 7.1 for me. It is not a reliable way to determine there is a change in processing. It is set to 7 in the config.

[NickM-27]

It changes from 7 to 7.1 for me. It is not a reliable way to determine there is a change in processing.

It'll go much higher than that when it's sending more frames like during a rain storm

[silentnyte]

I logged these for quite some time through various clear and light snowing and I never saw much of a change in these values. "camera_fps", "process_fps", "skipped_fps", "detection_fps" Right now it is currently lightly snowing and you can see here the effect it has on the CPU.

But the MQTT message you would not be able to detect the change.

[NickM-27]

Your screenshot doesn't even show detect_fps

[silentnyte]

My bad in most of the message I was looking at it was 0 so I was ignoring it. I should have thought about plotting along side of the CPU.

[silentnyte]

What would be the most effective way to reduce the object detection? detection_fps motion_threshold motion_contour_area If object detection is completely disabled then detection_fps would stay 0 and would not be able to detect when the storm has passed and turn detection back on.

[NickM-27]

I would increase contour_area, requiring that a larger part of the image must change to be clarified as motion.

[silentnyte]

During a storm the whole image is changing.

[NickM-27]

Usually just streaks right? In my cameras case it's just lots of dots and not the entire image all the time.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[VoyteckPL]

Did you find a solution for this?

[silentnyte]

I did not find a robust way to reduce the detection CPU utilization. Reducing the Detect_FPS or contour area did not have the effect that I was looking for. I played with just disabling detection when I detected bad weather. But in the end I am just living with the increased CPU utilization during bad weather.