[Support]: Motion frames on frigate startup cause object detection to "stick" on all parked cars in the area. Driving high detector usage.

[aredon]

Describe the problem you are having

Pretty much title. I upgraded to V13 and have been going through my cameras to update the configs and noticed that object detection is stuck on all the parked cars on my street that are in frame. As far as I can tell these just keep detecting until object detection is toggled off.

From what I can tell there are some errant motion boxes being drawn when frigate first starts. These are driving object detection for most of the frame - and then it just stays on because the object hasn't left.

Version

0.13.1-34FB1C2

Frigate config file

#GLOBAL FFMPEG
ffmpeg:
  output_args:
    record: preset-record-generic-audio-aac

#GLOBAL OBJECT SETTINGS
#Objects list: https://docs.frigate.video/configuration/objects/
objects:
  track:
    - person
    - dog
    - cat
    - bird
    - car

snapshots:
  retain:
    default: 7

birdseye:
  enabled: True
  mode: motion

#CAMERAS
cameras:
  ##################################################### OFFICE
  office:
    ffmpeg:
      inputs:
        - path: rtsp://Frigate:REDACTED@192.168.1.201:554//cam/realmonitor?channel=1&subtype=1
          roles:
            - detect
        - path: rtsp://Frigate:REDACTED@192.168.1.201:554//cam/realmonitor?channel=1&subtype=0
          roles:
            - record
    zones:
      porch:
        coordinates: 74,251,302,336,640,425,640,480,535,480,406,480,298,480,180,480,150,452,64,370,0,293,68,295
    detect:
      enabled: True
      # width: 640
      # height: 480
    record:
      enabled: True
      retain:
        days: 7
        mode: motion
      events:
        required_zones: 
          - porch
        retain:
          default: 7
          mode: active_objects
    snapshots:
      enabled: True
      bounding_box: True
      required_zones: 
        - porch
    motion:
      mask:
        - 211,86,640,36,640,480,573,480,491,480,482,420,465,364,469,292,375,271,285,254,277,325,246,328,181,300,66,262,71,137
        - 640,451,640,480,382,480,383,454

Relevant log output

N/A

FFprobe output from your camera

N/A

Frigate stats

No response

Operating system

Debian

Install method

HassOS Addon

Coral version

CPU (no coral)

Network connection

Wired

Camera make and model

Misc Amcrest Devices

Any other information that may be helpful

No response

[NickM-27]

As far as I can tell these just keep detecting until object detection is toggled off.

That's not how frigate works. Frigate knows when objects are stationary and doesn't run detection continuously (only periodically)

[NickM-27]

I'd suggest providing a screenshot of the system page

[aredon]

As far as I can tell these just keep detecting until object detection is toggled off.

That's not how frigate works. Frigate knows when objects are stationary and doesn't run detection continuously (only periodically)

Not sure what to tell you but that's what's happening. The whole debug frame is full of object bounding boxes with no motion boxes to speak of after the first couple frames. First few seconds of frames there's a red motion box over literally 95%+ of the frame, object bounding boxes show up right after that and stay highlighted in perpetuity with no motion boxes drawn on any subsequent frames for at least the ten minutes that I let it run. I am currently bringing up the motion threshold to see if that helps avoid the whole-frame motion box on startup - just sucks that I might not be able to go below whatever that ends up being.

[NickM-27]

Not sure what to tell you but that's what's happening. The whole debug frame is full of object bounding boxes with no motion boxes to speak of after the first couple frames.

yes, but that doesn't mean object detection is running continuously. Frigate tracks objects and knows they are there and only looks for objects if there is motion or on an interval. This is covered in the docs https://docs.frigate.video/configuration/stationary_objects

[aredon]

Not sure what to tell you but that's what's happening. The whole debug frame is full of object bounding boxes with no motion boxes to speak of after the first couple frames.

yes, but that doesn't mean object detection is running continuously. Frigate tracks objects and knows they are there and only looks for objects if there is motion or on an interval. This is covered in the docs https://docs.frigate.video/configuration/stationary_objects

I'm familiar with stationary objects - but perhaps I am not understanding how they behave. I would not expect bounding boxes to have shifting percentages, colors, and shapes if object detection is not running. I can also literally hear the cooling fan on my server chugging.

CPU detector is at 253% right now while this is going on. Normally this floats around 20%

[aredon]

[NickM-27]

most cameras are not running detection at all. Some have relatively low detect fps, but the high CPU usage is because you are using a cpu detector which is not recommended. You can watch the debug live view with regions enabled and that will show what detection is being done, detection would only be running constantly if there is motion or the bounding boxes are not consistent enough for the object to be considered stationary.

You have a lot of skipped fps because the inference speed of your CPU is too high so that is likely causing important detections to be skipped and making the problem much worse

[aredon]

Most cameras are not pointing outside so don't see much action. It's front door and office that can see the street. I have been running for months with this CPU setup in my VM without any skipped frames or high usage (I've been monitoring very closely). It's only after the update I've had problems.

Here's how it looks once I have toggled detection off and then back on again:

No detection is occurring because nothing is happening in frame. It's only after startup (or probably any motion that crosses the frame) that this skyrockets.

I will try restarting the server again and paying attention to regions instead of bounding boxes.

[aredon]

Ok after a restart I can confirm there are large detection regions drawn over most of the frame. There is no red motion boxes happening and the detections regions are jumping around a little. This behavior persists for a pretty long time - is there a timeout period I should be looking for?

[NickM-27]

right, on startup frigate runs detection to get an initial idea of what objects are present and did not just appear when motion is first detected.

by default an object box needs to be similar for 50 frames and then it will be considered stationary. If no motion is occurring and detection continues that is because the object box is not consistent. Or it can be because your CPU detector is causing skipped frames so the object is presumed missing

[aredon]

Perhaps I am not being clear. Here is the order of events after startup: 1.) Large red motion box covers 95%+ of the frame 2.) Object detection regions appear 3.) Motion box goes away 4.) Detection regions persist for 10+ minutes

The issue stops if I set the motion threshold to 120 for this camera. Which seems a bit high? Do you know what the threshold was in v12?

It does seem likely that skipped frames are part of the problem. Maybe I need to crank up the motion threshold for all my cameras? I am seeing some persistent motion boxes on some of the other cameras. So that may actually be the bulk of the CPU usage.

It just kind of sucks that I am setting motion threshold around startup behavior rather than day-to-day motion.

[NickM-27]

I don't understand. You say:

3.) Motion box goes away 4.) Detection regions persist for 10+ minutes

but then later say:

I am seeing some persistent motion boxes on some of the other cameras.

so does motion stop or does it keep going?

if motion is ongoing when nothing is changing then that means motion is too sensitive.

[aredon]

Apologies, there are two sets of cameras with different issues. Group 1 is my main concern. Group 1: Outdoor cameras with a persistent object detection issue. No motion detected except for startup. Group 2: Indoor cameras with motion detection at startup. This appears to taper down over a few minutes and then stop. It's unclear to me if that means the motion threshold needs dialed up or not.

[aredon]

Ok. Motion has settled on all the other cameras after being started for about 5 minutes, but, something moved on the front_door camera and now the detection region is stuck on as before. There are no skipped frames and no motion.

[NickM-27]

Might be good to see a screen recording of the debug view while this is happening so I can understand what is going on exactly

[aredon]

I can absolutely arrange that. Give me a minute.

[aredon]

This should be pretty good. https://github.com/blakeblackshear/frigate/assets/1328757/357a6582-4f46-4283-ac5e-25fc6eb3825f

Note that this camera's motion threshold is currently 110.

[NickM-27]

There is no motion going on here, the problem is that the cars are not detected consistently so detection continues to be run. If you don't care about those cars being detected could use a min_area filter for cars.

[aredon]

What's causing the cars to not detect consistently?

[NickM-27]

the default model isn't trained on security camera images, we can see from the video that the cars are detected with a very low score. also some of them look like false positives and they are not actually cars

[aredon]

So the model is kind of alternating between detecting a car and not detecting a car and thus persists at trying to detect in that area? I'll give the min_area filter a try.

[kzgrey]

I believe I have this same problem. The end result is that I have one video recording in progress constantly. Disabling the Car detection and re-enabling it appears to fix the problem. Turning Car detection back on makes the problem occur again. Right now, on my Events tab, I see two "in progress" recordings for the same camera. I see the same boundary box flutter with the stationary cars.

[NickM-27]

An in progress event does not mean it is being recorded, depends on the config. There is also a docs section for handling parked cars

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.