[Support]: Motion detected in the wrong zone

[B0ndo2]

Describe the problem you are having

I created 2 zones (sidewalk and driveway). I set up driveway to detect only people while sidewalk to detect cars. What is happening now is that a car stopped on the driveway keep detected as car even without any motion or sometime with someone walking on the sidewalk You can see from the screeshot that the zone for the event is sidewalk Check the video too I am also using the blueprint automation and getting car notification when the car is on the driveway zone, ( I tried removing drive_way zone from the trigger zone and then I get notifications for car see screenshot)

Version

0.11.1-2eada21

Frigate config file

# frigate.yml
detectors:
  cpu1:
    type: cpu
mqtt:
  # Required: host name
  host: caha.duckdns.org
  # Optional: port (default: shown below)
  port: 1883
  # Optional: topic prefix (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  topic_prefix: frigate
  # Optional: client id (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  client_id: frigate
  # Optional: user
  user: frigate
  # Optional: password
  # NOTE: Environment variables that begin with 'FRIGATE_' may be referenced in {}.
  #       eg. password: '{FRIGATE_MQTT_PASSWORD}'
  password: xxx
  # Optional: interval in seconds for publishing stats (default: shown below)
  stats_interval: 60
rtmp:
  enabled: false

#motion:
#  contour_area: 100
#  delta_alpha: 0.2
#  frame_alpha: 0.2
#  frame_height: 80
#  threshold: 25

birdseye:
  # Optional: Enable birdseye view (default: shown below)
  enabled: False
  # Optional: Width of the output resolution (default: shown below)
  width: 1280
  # Optional: Height of the output resolution (default: shown below)
  height: 720
  # Optional: Encoding quality of the mpeg1 feed (default: shown below)
  # 1 is the highest quality, and 31 is the lowest. Lower quality feeds utilize less CPU resources.
  quality: 8
  # Optional: Mode of the view. Available options are: objects, motion, and continuous
  #   objects - cameras are included if they have had a tracked object within the last 30 seconds
  #   motion - cameras are included if motion was detected in the last 30 seconds
  #   continuous - all cameras are included always
  mode: continuous

cameras:
  driveway:
    zones:
      sidewalk:
        coordinates: 114,64,231,43,366,36,425,39,474,50,521,80,553,84,529,51,414,16,302,12,167,21,147,34
        objects:
          - car
      drive_way:
        coordinates: 274,47,441,62,508,76,562,189,550,260,562,286,429,352,0,352,30,207,108,66
        objects:
          - person

    objects:
      track:
        - person
        - car
      filters:
        person:
          mask:
            - 114,64,231,43,368,39,415,42,474,50,521,80,553,84,529,51,414,16,302,12,167,21,147,34
          threshold: 0.58
          min_area: 3200
#          max_area: 30320
    ffmpeg:
      inputs:
        - path: rtsp://xx:xx@192.168.6.140/2
          roles:
            - detect
        - path: rtsp://xx:xx@192.168.6.140/1
          roles:
            - record
    detect:    
      width: 640
      height: 352
      fps: 7  
    motion:
      mask:
        - 0,0,181,0,100,81,0,304
        - 640,0,473,0,489,65,571,100,640,191    
        - 234,14,399,18,504,45,464,0,161,1,151,27
    record:
      enabled: true
      retain:
        days: 0
      events:
        required_zones:
          - drive_way
          - sidewalk
        retain:
          default: 5
          mode: active_objects
    snapshots:
      enabled: True
      bounding_box: True
      required_zones:
        - drive_way
        - sidewalk
      timestamp: True      
      retain:
        default: 7

################################################ 
  front_porch_camera:
    objects:
      track:
        - person
      filters:
        person:
          mask:
            - 148,348,171,122,173,12,377,8,518,57,640,80,640,0,0,0,0,352         
    ffmpeg:
      inputs:
        - path: rtsp://xx:xx@192.168.6.141/2
          roles:
            - detect
        - path: rtsp://xx:xx@192.168.6.141/1
          roles:
            - record
    detect:    
      width: 640
      height: 352
      fps: 5  
    motion:
      mask:
        - 148,348,171,122,173,12,377,8,518,57,640,80,640,0,0,0,0,352         
    record:
      enabled: true
      retain:
        days: 0
      events:
        retain:
          default: 5
          mode: motion
    snapshots:
      enabled: True
      bounding_box: True
      timestamp: True      
      retain:
        default: 7

Relevant log output

NA

FFprobe output from your camera

ffprobe version 5.1.3 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 12 (GCC)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --enable-chromaprint --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libbs2b --enable-libcdio --enable-libdrm --enable-libjack --enable-libjxl --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libilbc --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-libplacebo --enable-librsvg --enable-librav1e --enable-librubberband --enable-libsmbclient --enable-version3 --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libshaderc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-avfilter --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, rtsp, from 'rtsp://xx:xx@192.168.6.140/2':
  Metadata:
    title           : 11
  Duration: N/A, start: 0.100000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 640x352, 20 fps, 10 tbr, 90k tbn

Frigate stats

No response

Operating system

HassOS

Install method

HassOS Addon

Coral version

CPU (no coral)

Network connection

Wireless

Camera make and model

Chortau

Any other information that may be helpful

https://github.com/blakeblackshear/frigate/assets/75389943/de49b8f6-6c06-480f-af45-574d93351484

[NickM-27]

This is most likely due to cars driving by or other false positives being detected as a car and then the IDs being swapped making frigate think the car that drove by is the same as the car in the driveway. This is something that is being worked on in 0.13 along with a feature that will make it easy to see key moments like which point an object entered a zone.

Also, zones are for objects not motion to be clear.

[B0ndo2]

As you can see from the camera view, cars driving by can not be picked up by the camera (unless they go into the top end of the driveway which is marked as sidewalk). The car is re-detected when the light of a passing by car shines on it. I understand that zones are for objects and not motion but my configuration (as far as I understand) requests car object detection only inside the lower part of the driveway. Sometimes a person walking on the sidewalk re-triggers car detection too

Also in the snapshot there is no bounding box even though it is enabled and works with person

[NickM-27]

As you can see from the camera view, cars driving by can not be picked up by the camera (unless they go into the top end of the driveway which is marked as sidewalk).

I see in the top right a car could at least partially be seen. And since zone presence is baed on the bottom center pixel of the bounding box it could definitely be seen entering the sidewalk.

I understand that zones are for objects and not motion but my configuration (as far as I understand) requests car object detection only inside the lower part of the driveway.

no, that is incorrect. Objects are always detected everywhere, they are just not creating events unless they are in the required_zone.

Also in the snapshot there is no bounding box even though it is enabled and works with person

this is because the event is in progress, bounding boxes are annotated once the event finishes

[B0ndo2]

As you can see from the camera view, cars driving by can not be picked up by the camera (unless they go into the top end of the driveway which is marked as sidewalk).

I see in the top right a car could at least partially be seen. And since zone presence is baed on the bottom center pixel of the bounding box it could definitely be seen entering the sidewalk.

Which photo (I don't see any car other than mine)?, I see my car outside the driveway zone. This is not one incident.

I understand that zones are for objects and not motion but my configuration (as far as I understand) requests car object detection only inside the lower part of the driveway.

no, that is incorrect. Objects are always detected everywhere, they are just not creating events unless they are in the required_zone.

That is what I am saying, it is creating an event when it is in the zone which I marked as person only

Also in the snapshot there is no bounding box even though it is enabled and works with person

this is because the event is in progress, bounding boxes are annotated once the event finishes

Which means a box will never be shown in this case

[NickM-27]

Which photo (I don't see any car other than mine)?, I see my car outside the driveway zone. This is not one incident.

Please slow down and read what I am saying. I am saying that in the photos, I see areas in the top right which a car would be visible driving by which would also have its bottom center pixel in the sidwalk zone, that is likely when this happens. Then that car quickly disappears and frigate thinks the car in the driveway is the same car that briefly drove by. This is going to be fixed in 0.13.

That is what I am saying, it is creating an event when it is in the zone which I marked as person only

See above explanation

Which means a box will never be shown in this case

No? It just won't be shown until the car leaves.

[B0ndo2]

Any suggestions for a workaround for now ?

[NickM-27]

maybe try to make the top zone less close to the road. after all like it says in the docs, zone presence is based on the bottom center pixel of the bounding box so it doesn't need to be that close to the road to only catch when the car is leaving

[B0ndo2]

I just did that, let's see what happens. I want to make sure the car gets detected entering or leaving

[B0ndo2]

Now this is happening. No cars, but people are walking by triggering car notification

[NickM-27]

There's not enough context to understand what you're referring to. Sounds more like notifications are just incorrectly configured.

[B0ndo2]

This is a n event in the video, if you think the notifications are incorrectly configured; can you tell me where I went wrong ?

Look at the video below, notice the time transitions and there is no activity. This is for the car obect (driveway cam, drive_way zone)

[NickM-27]

"Activity" can be interpreted different ways. Based on your config recordings only keep segments with active objects for this camera so frigate saw an object that was active. May have been a false positive or incorrectly saw the car move.

[B0ndo2]

There are multiple case and repeatable so being a false positive doesn't sound like the issue. I have a lot of cases where someone walking on the sidewalk triggers car detection.

Shouldn't this limit it ?

  drive_way:
    coordinates: 274,47,441,62,508,76,562,189,550,260,562,286,429,352,0,352,30,207,108,66
    objects:
      - person

[NickM-27]

This is where I think your notifications are incorrect. Does it ever say the car is actually in the driveway zone? In any case without more info like notification setup, a picture of the events view, etc we can only guess why this is happening.

[B0ndo2]

I am using Frigate notification blueprint and I posted a screen shot of it up there. Here it is again

I tried to put the video but it is too big, I see segments of video added to the recording of a car when a person walks

What else can I provide ?

[NickM-27]

the cars have been detected in the sidwalk zone, not the drive_way. So the zones are working as expected. Based on the cars moving around in the snapshot t looks like the events are working as expected, I don't use the blueprint so not 100% sure why a person would be triggering the car event but that is a question for the blueprint dev.

I also just realized that you are running 0.11, I'd definitely suggest upgrading to 0.12

[B0ndo2]

Are there changes in 0.12 related to this issue ?

The cars are stationary but when a person moves around them it records a segment, is that expected ?

[NickM-27]

Are there changes in 0.12 related to this issue ?

Not specifically but it makes it easier to debug especially since the latest blueprint is designed to work with 0.12 specifically. Also makes it easier to upgrade to 0.13 when that eventually becomes available

The cars are stationary but when a person moves around them it records a segment, is that expected ?

Yes, that is expected

[B0ndo2]

The cars are stationary but when a person moves around them it records a segment, is that expected ?

Yes, that is expected

Even if the zone has a filter for person only (the recording segment is added to both the car and person video) ?

The notification blueprint is referred to in the docs. here

If detection is based on the bottom center of the object then the car's hood being in the sidewalk zone wouldn't matter or would it (I am referring to the car on the left in the recent post where people were walking on the sidewalk and a notification was triggered) ?

[B0ndo2]

All I need is to record and get notifications only a car moves in or out of the driveway and people anywhere except the sidewalk

[NickM-27]

The cars are stationary but when a person moves around them it records a segment, is that expected ?

Yes, that is expected

Even if the zone has a filter for person only (the recording segment is added to both the car and person video) ?

Yes, I would highly recommend reading the docs. The behavior for active_objects recording mode is simple. If there is ANY object which is active (not stationary) then that recording segment will be saved. Then when you play back the event for the car any recording segments that exist for that time period that the car was visible will be shown.

The notification blueprint is referred to in the docs. here

Yes, that is designed for 0.12

If detection is based on the bottom center of the object then the car's hood being in the sidewalk zone wouldn't matter or would it (I am referring to the car on the left in the recent post where people were walking on the sidewalk and a notification was triggered) ?

Correct, the cars hood won't matter. We still have not seen an example of one of these notifications which would be helpful as well.

[NickM-27]

Also to be clear we have had this same conversation before https://github.com/blakeblackshear/frigate/issues/4977#issuecomment-1376032051

[B0ndo2]

If detection is based on the bottom center of the object then the car's hood being in the sidewalk zone wouldn't matter or would it (I am referring to the car on the left in the recent post where people were walking on the sidewalk and a notification was triggered) ?

Correct, the cars hood won't matter. We still have not seen an example of one of these notifications which would be helpful as well.

How do I provide an example of the notification?. I receive notifications on my phone and I go look at the video and there is only people walking close to the car. The automation is only for cars.

I can clearly see clips where a car's headlight shines on the car on the driveway and I get a notification

[B0ndo2]

I upgraded to 0.12 2 days ago and I haven't had the issue since. No clips with people moving around the car in the car zone. Looks like there was a change in 0.12. I will keep this open for a couple of days more. Thank you so much for your help

[B0ndo2]

I am seeing something different now. In the event view, the thumbnail of the event shows a different image than the video and also the time is different

[B0ndo2]

Now I have this,the event showing in the logbook but the automation blueprint didn't fire at all (I enabled debug too). The event was recorded

[NickM-27]

I am seeing something different now. In the event view, the thumbnail of the event shows a different image than the video and also the time is different

that is of course expected, the thumbnail / snapshot is not from the beginning of the event

[B0ndo2]

I am seeing something different now. In the event view, the thumbnail of the event shows a different image than the video and also the time is different

that is of course expected, the thumbnail / snapshot is not from the beginning of the event

The thumbnail is not in any part of the recorded video, am confused

[NickM-27]

what is confusing? the snapshot can come from any part of the objects lifespan based on certain parameters. You have active_objects set for recordings but of course that only affects recordings and not snapshots

[B0ndo2]

What about the notification not firing now ?

[NickM-27]

That would be a question for the blueprint dev or the HA automation trace.

[B0ndo2]

Could it be that the messages are not registering in MQTT ? Is there a way to look back at that time ?

[NickM-27]

The HA trace will show what messages were received. There is no way to view mqtt history from before but if you open mqtt explorer it will save history while it is open

[B0ndo2]

Do you mean this ? I checked automation traces and the automation didn't even start. Last automation run 12:05:35PM Car detected 12:29:48PM

[B0ndo2]

I think i know now where my problem was. I set "Update Attachment (Optional)" on in the blueprint. This was causing a new notification on my phone when a person walks by the car on the sidewalk zone. I disabled the option and I am no longer getting those notifications but I still see the people walking by in the events (car filter). Not sure if this is by design.

[B0ndo2]

I was wrong, the problem is still there. Anyone walking in the sidewalk zone while the car is stationary in the driveway zone triggers an event of type car in sidewalk zone

[blakeblackshear]

This is expected, but you are describing what is happening wrong.

1. A person walks down the sidewalk
2. Frigate detects motion
3. Frigate crops a portion of the image that includes the areas with motion. Because the car is close enough to the sidewalk, it is also within the portion of the image where Frigate runs object detection.
4. Frigate runs object detection
5. All objects detected in the image are found and Frigate starts tracking them with an event until they are no longer detected.
6. An event is created for the person, AND any other objects that were detected in the areas where object detection ran.

There is no way to know which object actually caused the motion.

Was the car already actively being tracked (did it have a bounding box already in debug view) by Frigate when the person walked by?

[blakeblackshear]

I see now that you have a sidewalk zone specifically for car and your car events are marked with the sidewalk zone. I would suggest setting a reasonable min_area filter for car on the sidewalk zone. My guess is that you have a single frame where a car is being detected as a false positive in that zone.

[B0ndo2]

I see now that you have a sidewalk zone specifically for car and your car events are marked with the sidewalk zone.

That is exactly my point. The car is stationary in the driveway zone, the 2 zones do not overlap at all. The bottom center of the car's bounding box is completely inside the driveway zone, the events show that the confidence is 80+%.

The image below shows one of those cases and what makes it even more difficult is that there is no bounding box with the event

[blakeblackshear]

At some point during that event, there was a frame with a car (likely false positive) detected within the sidewalk zone that Frigate thought was the same as the parked car. In 0.13, you will be able to see the exact bounding box.

My best guess is that the car driving by on the street in that image had the bottom center of it's bounding box in the sidewalk zone and when it left, Frigate assumed the car in your driveway was the same car. I would suggest making a driveway_entrance zone rather than using the sidewalk zone for cars. I would expect nearly every car driving by on the street to enter the sidewalk zone based on how you have drawn it.

Something like this:

[B0ndo2]

At some point during that event, there was a frame with a car (likely false positive) detected within the sidewalk zone that Frigate thought was the same as the parked car. In 0.13, you will be able to see the exact bounding box.

Waiting for 0.13

My best guess is that the car driving by on the street in that image had the bottom center of it's bounding box in the sidewalk zone and when it left, Frigate assumed the car in your driveway was the same car. I would suggest making a driveway_entrance zone rather than using the sidewalk zone for cars. I would expect nearly every car driving by on the street to enter the sidewalk zone based on how you have drawn it.

This is not possible as my sidewalk zone is well far from the street so for a car to be there it means that the driver was actually driving on the sidewalk

[NickM-27]

This is not possible as my sidewalk zone is well far from the street so for a car to be there it means that the driver was actually driving on the sidewalk

Not really, the box would only have to be slightly lower than the bottom of the car for it to be in the zone (remember, the bottom center of the box is all that matters for the zone)

[NickM-27]

You might find some better luck moving the zone to only cover the part where the car will drive and not the grass to the right and left.

[B0ndo2]

You might find some better luck moving the zone to only cover the part where the car will drive and not the grass to the right and left.

I will definitely do that , looks like I might need to create a third zone by splitting the sidewalk into 2 zones. One for the actual sidewalk and one for the extended part of the driveway

[B0ndo2]

This is not possible as my sidewalk zone is well far from the street so for a car to be there it means that the driver was actually driving on the sidewalk

Not really, the box would only have to be slightly lower than the bottom of the car for it to be in the zone (remember, the bottom center of the box is all that matters for the zone)

Are you suggesting that I move the upper border of my zone down ?

[NickM-27]

Yeah, I like to have the one for driveway called driveway_entrance. And no I'm not necessarily suggesting that, a lot of the 0.13 changed will help a lot with that

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.