[Config Support]: Required Zones bypassed only with MQTT

[Morphy99]

Describe the problem you are having

I'm trying to eliminate false positives with one of my cameras and I have set up filters and required zones as per the docs. MQTT is still occasionally reporting Person detected however this isn't listed in events.

I've set the person filter as threshold 0.8 in Zones and set required_zones in MQTT, Snapshots, and record,events.

Version

0.11.0-37325C7

Frigate config file

mqtt:
  host: xxxxx

database:
  # The path to store the SQLite DB (default: shown below)
  path: /db/frigate.db

detectors:
  coral_pci:
    type: edgetpu
    device: pci

objects:
  track:
    - person
    - car
    - cat
    - dog
  filters:
    person:
      threshold: 0.7

record:
  enabled: True
  retain:
    days: 0
  events:
    retain:
      default: 20
      mode: active_objects
    pre_capture: 10
    post_capture: 10

snapshots:
  enabled: True
  timestamp: False
  bounding_box: True
  crop: False
  retain:
    default: 20

###############################################################back patio######################################################################

  back_patio:
    ffmpeg:
      input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -rw_timeout 5000000
      output_args: 
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
      inputs:
        - path: http://xxxxxxx/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=xxxx&password=xxxx
          roles:
            - record
            - rtmp
        - path: http://xxxxxxxx/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=xxxx&password=xxxx
          roles:
            - detect
    detect:
      width: 896
      height: 672
      fps: 7
    mqtt:
      required_zones:
        - back_patio_left
        - back_patio_right
        - back_patio_steps
        - back_patio_shed
    live:
      height: 1080
      quality: 15
    record:
      events:
        retain:
          default: 10
        objects:
          - person
          - cat
          - dog
        required_zones:
          - back_patio_left
          - back_patio_right
          - back_patio_steps
          - back_patio_shed
    snapshots:
      required_zones:
        - back_patio_left
        - back_patio_right
        - back_patio_steps
        - back_patio_shed
    objects:
      track:
        - person
        - cat
        - dog
    zones:
      back_patio_left:
        coordinates: 648,0,322,0,320,303,623,291
        filters:
          person:
            min_area: 10000
            max_area: 50000
            threshold: 0.8
      back_patio_right:
        coordinates: 195,672,896,672,896,246,557,285,295,290
        filters:
          person:
            min_area: 30000
            threshold: 0.8
      back_patio_steps:
        coordinates: 214,531,146,156,0,152,0,48,351,59,256,532
        filters:
          person: 
            min_area: 75000
            threshold: 0.8
      back_patio_shed:
        coordinates: 122,149,230,525,197,672,0,672,0,174
        filters:
          person:
            min_area: 100000
            threshold: 0.8
    motion:
      mask:
        - 396,0,397,46,0,42,0,0
        - 326,41,256,144,133,108,128,40

Relevant log output

NA nothing relevant in logs

Frigate stats

NA

Operating system

Debian

Install method

Docker CLI

Coral version

PCIe

Any other information that may be helpful

No response

[blakeblackshear]

Which mqtt topic?

[NickM-27]

MQTT -> required_zones only restricts the MQTT snapshots from being sent.

If you're referring to the /events topic in MQTT, that will always send all updates and you need to filter based on the zones fields that are included if you only want updates when an object is in a zone: https://docs.frigate.video/integrations/mqtt#frigateevents

[Morphy99]

Which mqtt topic?

I've just checked my HA automation and it's triggered from the sensor.camera_X_person entity. I was assuming this was triggered by MQTT in the Frigate HA integration?

[NickM-27]

Which mqtt topic?

I've just checked my HA automation and it's triggered from the sensor.camera_X_person entity. I was assuming this was triggered by MQTT in the Frigate HA integration?

It is, required_zones doesn't apply to it though. You'd need to use the sensor.zone_X_person entity for that.

[Morphy99]

MQTT -> required_zones only restricts the MQTT snapshots from being sent.

If you're referring to the /events topic in MQTT, that will always send all updates and you need to filter based on the zones fields that are included if you only want updates when an object is in a zone: https://docs.frigate.video/integrations/mqtt#frigateevents

Ah OK I didn't realise that. My automation is off the person sensor in the HA integration. Would it work as well if I set the threshold in the camera under object, person, filter, as I want all zones the same threshold?

    objects:
      track:
        - person
        - cat
        - dog
      filters:
        person:
          threshold: 0.8

[Morphy99]

Which mqtt topic?

I've just checked my HA automation and it's triggered from the sensor.camera_X_person entity. I was assuming this was triggered by MQTT in the Frigate HA integration?

It is, required_zones doesn't apply to it though. You'd need to use the sensor.zone_X_person entity for that.

OK I see. I was under the impression if the event threshold wasn't met, it wouldn't publish via mqtt frigate/events

[NickM-27]

Threshold is different than zones and would be a separate discussion. But if your goal is to only react if a person is in one of your zones, you'd either want to use the RAW MQTT topic for the trigger or use each of the zone specific sensors and allow any of them to trigger the automation (but make sure it doesn't fire multiple times if a person walks through multiple zones at once).

[Morphy99]

My issue is trying to eliminate false positives on this camera. As it's a small area that's covered and people are large in the frame, positive detections are a high probability so I need the threshold around 0.8. I was getting a few false positives around the default 0.7 as the area is quite busy visually and constantly changing, masks aren't an option.

I already have an automation for the person in zone MQTT trigger which sends the snapshot and clip links. There is a slight delay with this one though as it has to wait for the end of the event.

  trigger:
    platform: mqtt
    topic: frigate/events
    payload: end
    value_template: "{{ value_json.type }}"
  condition:
    - "{{ 'person' in trigger.payload_json['after']['label'] }}"
    - condition: or
      conditions:
        - "{{ 'front_drive' in trigger.payload_json['after']['entered_zones'] }}"
        - "{{ 'front_path' in trigger.payload_json['after']['entered_zones'] }}"
        - "{{ 'front_door' in trigger.payload_json['after']['camera'] }}"
        - "{{ 'back_patio' in trigger.payload_json['after']['camera'] }}"
        - "{{ 'back_garden' in trigger.payload_json['after']['camera'] }}"

I want one which triggers as soon as the object is positively detected, so I have another:

  trigger:
  - platform: numeric_state
    entity_id: sensor.front_drive_person
    above: 0
  - platform: numeric_state
    entity_id: sensor.front_path_person
    above: 0
  - platform: numeric_state
    entity_id: sensor.back_garden_person
    above: 0
  - platform: numeric_state
    entity_id: sensor.back_patio_person
    above: 0

This is where I was having issues as these sensors were triggering despite using required_zones and person filters for those zones. As I mentioned in my previous post I have now set a filter for objects in this camera and the first night I didn't get any false positives so hopefully this has fixed it.

Using Zone sensors would be an alternative but as you mention, multiple triggers would be an issue. Speaking of which, I have had that problem with using the sensor trigger. My automation would trigger 3-4 times if the person remained in frame for a while. Is there some sort of "debounce" I can use or do you have any examples of automation logic on hand to eliminate this?

[NickM-27]

You could either use the built in HA debounce or just use the MQTT trigger natively as it gives way more info. I think it may make sense in a future version to have a denounce option in frigate

[Morphy99]

I'm still getting some weird behaviour with my automation triggering despite the filters set up. Last night I had a "person in zone" event which wasn't in the Frigate database and there was no snapshot or clip associated with it at the time. In fact the only evidence I can now find was a screenshot of the notification I took at the time as I didn't really want to get up and start looking into it at 2am!

The first notification at 4mins is the one triggered off the sensor.back_patio_person and the second one on 3mins is from the mqtt topic with "end" payload. The threshold, it seems, was met but there was no clip or snapshot:

I've checked the docker logs for frigate and nothing at that time either. There are two clips slightly after the time of the notification which was around 02:41 but these are at 02:48 and only last 10 seconds.

Do I need to enable debug logging to try and figure out what's going on here?

[blakeblackshear]

Was there another event at that same time? There is some cleanup logic to remove events that appear to be duplicates. For example, if two people walk by together, you will get two separate events of almost identical length because there are two objects. One of those would be deleted a few minutes later in the cleanup routine.

[Morphy99]

Was there another event at that same time?

Nope, nothing on any camera until 02:47.

[blakeblackshear]

Do your notifications check for the required zones before notifying?

[Morphy99]

Do your notifications check for the required zones before notifying? The automations are here and for this particular camera there is no required zone in the mqtt condition. However, I've got required_zones set in mqtt at camera level. In any case, this latest issue may be different to the OT, it may have met the condition criteria, I can't verify because apparently it didn't happen according to Frigate!

[NickM-27]

Your conditions have or and you're checking for the camera name. Meaning your camera saw an object that never entered a required zone (so no frigate event) but you're automation fired because it was on the back porch camera

condition:
    - "{{ 'person' in trigger.payload_json['after']['label'] }}"
    - condition: or
      conditions:
        - "{{ 'front_drive' in trigger.payload_json['after']['entered_zones'] }}"
        - "{{ 'front_path' in trigger.payload_json['after']['entered_zones'] }}"
        - "{{ 'front_door' in trigger.payload_json['after']['camera'] }}"
        - "{{ 'back_patio' in trigger.payload_json['after']['camera'] }}"
        - "{{ 'back_garden' in trigger.payload_json['after']['camera'] }}"

[NickM-27]

If you want the zones to be a required condition for that automation it needs to be separate from that or

[Morphy99]

But I have this in the frigate config for that camera:

    mqtt:
      required_zones:
        - back_patio_left
        - back_patio_right
        - back_patio_steps
        - back_patio_shed

[NickM-27]

Right, as was said earlier mqtt -> required_zones only restricts the snapshots from being sent. It does not restrict the /events topic from being updated. If you want to filter on events for MQTT events topic it needs to be done in the automation

[Morphy99]

Ahhhhh, yes sorry! I forgot that crucial snippet. Maybe you need to look at renaming that config entry mqtt_snapshots to avoid confusion. I'll adjust my automation conditions to require zones.