[Support]: Masks are not loaded quickly enough during restart - Events are triggered

bortek commented 1 year ago

Describe the problem you are having

My Frigate is integrated to HASS and using https://github.com/blakeblackshear/frigate-hass-integration and I have setup notifications with the help of this simple Automation. It sends a notification to my Telegram whenever an event is triggered attaching an image.

alias: Frigate Notification
description: ""
trigger:
  - platform: mqtt
    topic: frigate/events
    payload: new
    value_template: "{{ value_json.type }}"
action:
  - delay:
      hours: 0
      minutes: 0
      seconds: 4
      milliseconds: 0
  - service: notify.telegram_notify
    data:
      message: MSG
      title: Tittle
      data:
        photo:
          - url: >-
              http://192.168.2.194:8123/api/frigate/notifications/{{trigger.payload_json["after"]["id"]}}/snapshot.jpg
            caption: >-
              A {{trigger.payload_json["after"]["label"]}} was detected by
              {{trigger.payload_json["after"]["camera"]}} camera.
mode: single

Due to my Reolink cameras often hang I need to restart Frigate container every 30 minutes. The problem is that during the startup of the container what I experience is that the masks are NOT loaded quickly enough before the object is detected. Or rather masks are not loaded before the events "engine" starts and kicks in.

Therefore if there is an object detected during the Frigate startup the result will be the following:

Frigate starts (masks are not loaded yet)
Object enters camera area
Object is detected
Event is triggered which triggers notification
Masks are loaded

This is my theory but I pretty sure it is the case because the evens are triggered exactly at every 30 minutes time intervals like 2:30, 3:00, 5:30 etc.

Any idea what can be done about this?

Version

0.11.1-2EADA21

Frigate config file

mqtt:
  host: 192.168.2.194
  user: frigate
  password: 

detectors:
  coral:
    type: edgetpu
    device: usb

cameras:
  jo-frontside-1:
    ffmpeg:
      hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
      input_args: -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rw_timeout 5000000
      inputs:
        # Ethernet
        #- path: rtmp://192.168.2.27/bcs/channel0_main.bcs?channel=0&stream=0&user=user&password=XXX
        - path: rtmp://192.168.2.176/bcs/channel0_main.bcs?channel=0&stream=0&user=user&password=XXX
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2560 # <---- update for your camera's resolution
      height: 1920 # <---- update for your camera's resolution
      fps: 25
    motion:
      mask:
        - 1083,349,1214,388,1647,428,2010,441,2295,465,2560,616,2560,0,1000,0

  jo-driveway-1:
    ffmpeg:
      hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
      input_args: -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rw_timeout 5000000
      inputs:
        # Ethernet
        # - path: rtmp://192.168.2.70/bcs/channel0_main.bcs?channel=0&stream=0&user=user&password=XXX
        - path: rtmp://192.168.2.150/bcs/channel0_main.bcs?channel=0&stream=0&user=user&password=XXX
          roles:
            - detect
            - record
            - rtmp
    detect:
      width: 2560 # <---- update for your camera's resolution
      height: 1440 # <---- update for your camera's resolution
      fps: 25
    motion:
      mask:
        - 1557,351,1261,359,1112,377,973,455,857,531,706,575,424,653,0,859,0,380,0,0,1577,0

rtmp: # Restream RTMP to e.g. HASS, not now.
  enabled: true

record:
  enabled: true
  retain:
    days: 20
    mode: motion
  events:
    pre_capture: 5
    post_capture: 5
    retain:
      default: 20
  expire_interval: 60

objects:
  track:
    - person
    - car
    - bicycle
    - bus
      #- refrigerator # värmepump
    - dog
    - bear
    - bird
    - cat
    - skis
    - snowboard

snapshots:
  enabled: true
  clean_copy: false
  timestamp: false
  bounding_box: true
  crop: False
  retain:
    default: 30

timestamp_style:
  position: "tl"
  format: "%Y-%m-%d %H:%M:%S"
  color:
    red: 255
    green: 255
    blue: 255
  thickness: 2

birdseye:
  enabled: True
  width: 1920
  height: 1130
  # 1 is the highest quality, and 31 is the lowest. Lower quality feeds utilize less CPU resources.
  quality: 5
  #   objects - cameras are included if they have had a tracked object within the last 30 seconds
  #   motion - cameras are included if motion was detected in the last 30 seconds
  #   continuous - all cameras are included always
  mode: continuous

database:
  path: /media/frigate/frigate.db

logger:
  default: info
  # Optional: Component specific logger overrides
  logs:
    frigate.event: debug
    frigate.record: debug

Relevant log output

N/A

FFprobe output from your camera

N/A

Frigate stats

No response

Operating system

Debian

Install method

HassOS Addon

Coral version

USB

Network connection

Wired

Camera make and model

N/A

Any other information that may be helpful

No response

blakeblackshear commented 1 year ago

Frigate scans the entire frame for objects on startup. Motion masks don't prevent objects from being detected. Please read through this section of the docs: https://docs.frigate.video/configuration/masks

bortek commented 1 year ago

Sorry I wasn't quite clear in the description perhaps. What I meant was that the motion kicks in directly at startup without taking into account the mask. Or else why would I have som many false positives every time frigate restarts?

bortek commented 1 year ago

I have enabled debug on all and will try to see what logs re saying during startup when they generate an event.

blakeblackshear commented 1 year ago

It's not motion triggered. Frigate scans the entire frame for objects to find any stationary objects at start up regardless of motion.

A "false positive" would be a tree being detected as a person. A person or car being detected somewhere you aren't interested in isn't a false positive. Can you be more specific about what you mean?

bortek commented 1 year ago

Let me try to elaborate :)

This is how my motion mask looks like and whats masked out should not trigger any motion detection, Right? And it does not. No cars are triggering motion when they pass only in that masked out area. See this pic

But if I restart frigate AND the happen to be a car in the masked out area only (not below, not above but only in the masked out area) then the event is being triggered, even though it should not because the car is only at the masked area.

This is the image that was triggered during 11 seconds after the Frigate restarted. You can clearly see the car being inside the masked out area and there are no other triggers in the picture. So my theory is that the masks are not being tolerated for some reason only during Frigate startup. Here is the proof. Look at the timestamp 07:30:11 on the image. Frigate restarted at 7:30:00

And here are the DEBUG logs from the container log. You can see at the exact same time an event is being triggered. Check my comments in-line.

I dont see anything in the logs related to motion detection (like starting motion detection, etc.) and I dont know precise logic of the startup process in relation to the motion masks but I looks like they are not being preloaded/started/used/etc fast enough only during startup.

One possible solution would be to delay motion detection process during Frigate startup by X seconds using a config variable (delay_startup_md=10 # Delay motion detection by number of seconds only during Frigate startup, default 0 seconds)

[s6-finish] sending all processes the TERM signal.               <--------- Here I restart Frigate using cronjob at 7:30:00 using cronjob
[s6-finish] sending all processes the KILL signal and exiting.
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2023-03-12 07:30:07] frigate.app                    INFO    : Starting Frigate (0.11.1-2eada21) <--------- Here Frigate starts, took 7 secs to stop it.
Starting migrations
[2023-03-12 07:30:07] peewee_migrate                 INFO    : Starting migrations
[2023-03-12 07:30:07] peewee                         DEBUG   : ('CREATE TABLE IF NOT EXISTS "migratehistory" ("id" INTEGER NOT NULL PRIMARY KEY, "name" VARCHAR(255) NOT NULL, "migrated_at" DATETIME NOT NULL)', [])
[2023-03-12 07:30:07] peewee                         DEBUG   : ('SELECT "t1"."id", "t1"."name", "t1"."migrated_at" FROM "migratehistory" AS "t1" ORDER BY "t1"."id"', [])
There is nothing to migrate
[2023-03-12 07:30:07] peewee_migrate                 INFO    : There is nothing to migrate
[2023-03-12 07:30:07] detector.coral                 INFO    : Starting detection process: 216
[2023-03-12 07:30:07] frigate.app                    INFO    : Output process started: 218
[2023-03-12 07:30:07] ws4py                          INFO    : Using epoll
[2023-03-12 07:30:07] frigate.app                    INFO    : Camera processor started for jo-frontside-1: 224
[2023-03-12 07:30:07] frigate.app                    INFO    : Camera processor started for jo-driveway-1: 227
[2023-03-12 07:30:07] frigate.app                    INFO    : Capture process started for jo-frontside-1: 229
[2023-03-12 07:30:07] frigate.app                    INFO    : Capture process started for jo-driveway-1: 230
[2023-03-12 07:30:07] urllib3.connectionpool         DEBUG   : Starting new HTTPS connection (1): api.github.com:443
[2023-03-12 07:30:07] frigate.mqtt                   DEBUG   : MQTT connected
[2023-03-12 07:30:07] frigate.mqtt                   DEBUG   : on_recordings_toggle: frigate/jo-frontside-1/recordings/set ON
[2023-03-12 07:30:07] frigate.mqtt                   DEBUG   : on_snapshots_toggle: frigate/jo-frontside-1/snapshots/set ON
[2023-03-12 07:30:07] frigate.mqtt                   DEBUG   : on_detect_toggle: frigate/jo-frontside-1/detect/set ON
[2023-03-12 07:30:07] urllib3.connectionpool         DEBUG   : https://api.github.com:443 "GET /repos/blakeblackshear/frigate/releases/latest HTTP/1.1" 200 1217
[2023-03-12 07:30:07] ws4py                          INFO    : Using epoll
[2023-03-12 07:30:07] peewee.sqliteq                 DEBUG   : received query UPDATE "event" SET "end_time" = ("event"."start_time" + ?) WHERE ("event"."end_time" IS ?)
[2023-03-12 07:30:07] peewee                         DEBUG   : ('UPDATE "event" SET "end_time" = ("event"."start_time" + ?) WHERE ("event"."end_time" IS ?)', [30, None])
[2023-03-12 07:30:07] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2023-03-12 07:30:10] frigate.edgetpu                INFO    : TPU found
[2023-03-12 07:30:11] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/jo-frontside-1/all.
[2023-03-12 07:30:11] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/jo-driveway-1/all.
[2023-03-12 07:30:11] frigate.events                 DEBUG   : Event received: start jo-driveway-1 1678602610.827524-4tn9fc <--------- Here we have an event created for a detected car, but it shouldn't have happened.
[2023-03-12 07:30:11] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/events.
[2023-03-12 07:30:11] frigate.events                 DEBUG   : Event received: update jo-driveway-1 1678602610.827524-4tn9fc
[2023-03-12 07:30:11] peewee.sqliteq                 DEBUG   : received query INSERT INTO "event" ("id", "label", "camera", "start_time", "end_time", "top_score", "false_positive", "zones", "thumbnail", "has_clip", "has_snapshot", "region", "box", "area", "retain_indefinitely", "ratio") VALUES (?, ?, ?, ?, ?, ?, ?, json(?), ?, ?, ?, json(?), json(?), ?, ?, ?)
[2023-03-12 07:30:11] peewee                         DEBUG   : ('INSERT INTO "event" ("id", "label", "camera", "start_time", "end_time", "top_score", "false_positive", "zones", "thumbnail", "has_clip", "has_snapshot", "region", "box", "area", "retain_indefinitely", "ratio") VALUES (?, ?, ?, ?, ?, ?, ?, json(?), ?, ?, ?, json(?), json(?), ?, ?, ?)', ['1678602610.827524-4tn9fc', 'car', 'jo-driveway-1', 1678602605.827524, None, 0.7265625, False, '[]', '/9j/4AAQSkZJRgABA----INTENTIONALLY_REMOVED_LONG_LINE----------wo25HapPOt8438Z/umlV4D/y0xn2NFxcp/9k=', True, True, '[899, 0, 1219, 320]', '[940, 62, 1185, 174]', 27440, False, 1.0])
[2023-03-12 07:30:11] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/jo-driveway-1/car/snapshot.
[2023-03-12 07:30:11] frigate.mqtt                   DEBUG   : MQTT payload for frigate/jo-driveway-1/car/snapshot wasn't text. Skipping...
[2023-03-12 07:30:11] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/jo-driveway-1/car.
[2023-03-12 07:30:11] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/jo-driveway-1/all.
[2023-03-12 07:30:16] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/events.
[2023-03-12 07:30:16] frigate.events                 DEBUG   : Event received: end jo-driveway-1 1678602610.827524-4tn9fc
[2023-03-12 07:30:16] peewee.sqliteq                 DEBUG   : received query UPDATE "event" SET "label" = ?, "camera" = ?, "start_time" = ?, "end_time" = ?, "top_score" = ?, "false_positive" = ?, "zones" = json(?), "thumbnail" = ?, "has_clip" = ?, "has_snapshot" = ?, "region" = json(?), "box" = json(?), "area" = ?, "ratio" = ? WHERE ("event"."id" = ?)
[2023-03-12 07:30:16] peewee                         DEBUG   : ('UPDATE "event" SET "label" = ?, "camera" = ?, "start_time" = ?, "end_time" = ?, "top_score" = ?, "false_positive" = ?, "zones" = json(?), "thumbnail" = ?, "has_clip" = ?, "has_snapshot" = ?, "region" = json(?), "box" = json(?), "area" = ?, "ratio" = ? WHERE ("event"."id" = ?)', ['car', 'jo-driveway-1', 1678602605.827524, 1678602621.689034, 0.796875, False, '[]', '/9j/4AAQSkZJR------INTENTINALLY-REMOVED_THE_REST-----gA+8fbK07isRFfwo25HapPOt8438Z/umlV4D/y0xn2NFxcp/9k=', True, True, '[892, 0, 1228, 336]', '[937, 64, 1183, 177]', 27798, 2.1769911504424777, '1678602610.827524-4tn9fc'])
[2023-03-12 07:30:17] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/jo-driveway-1/all.
[2023-03-12 07:30:17] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/jo-driveway-1/car.
[2023-03-12 07:30:17] frigate.mqtt                   DEBUG   : Received mqtt message on frigate/jo-driveway-1/car/snapshot.
[2023-03-12 07:30:17] frigate.mqtt                   DEBUG   : MQTT payload for frigate/jo-driveway-1/car/snapshot wasn't text. Skipping...

blakeblackshear commented 1 year ago

Again. This is expected behavior. As I have said multiple times and the docs also state, motion masks do not prevent objects from being detected. Motion is not being detected in that area. Frigate scans the entire frame on startup. It will begin tracking that car in case it later enters an area of interest.

Frigate is not designed to have areas you aren't interested in masked off that way. It is designed to have the areas you are interested in set as zones. You shouldn't be using a large motion mask like this as the docs explain. You should be creating zones for the areas you want to monitor and setting the zones as required for record and snapshots.

Even if it wasn't detected on startup, it would very likely be detected in the background when something else enters the unmasked area.

bortek commented 1 year ago

Thanks for the explanation. Entering in the unmasked area and triggering an event is fine and self explanatory. But the part about "...Frigate scans the entire frame on startup. It will begin tracking that car in case it later enters an area of interest..." is not so intuitive. Anyway, I have now setup the zone and will see how it behaves in the coming days. Cheers!

blakeblackshear / frigate