Frigate 0.14 - Githubissues

SgtBatten commented 4 months ago

Bugs

[x] sub_labels not defined
[x] "Person['person']" due to bad variable mapping
[x] Notifications labeled with the wrong camera
[ ] Only get alerts IF zone filter switch is OFF (Possibly fixed) SOMEONE PLEASE CONFIRM
[ ] Short events result in an incomplete video being attached which often doesnt show the objects (delay needed?)
[x] Turning on filters and selecting person causes all of my cameras NOT to send a notification.
[x] Delay in notifications arriving

Features

[x] user can set choose severity (alert or detection)
[x] can attach the new review gif to the notification (pending frigate integration API update)
[ ] incorporate new tts options from main branch @djak250

To Do:

[ ] ~~Update the loop to make it work with 0.14~~
[ ] confirm if the user choice of alerts/detections works
[ ] send message on end of event is video is meant to be send (best chance of recieving video)
[ ] one automation for multiple cameras (I have some ideas)
[ ] merge all changes that have been made in the main branch

Thoughts

Do we process it as a loop like the current frigate/events version, or process each message in frigate/reviews
Can it be built such that it supports both the review and event topics (user selection)

HenningL-Atea commented 4 months ago

Tested the latest version, and saving automation was successful, but automation had an error when triggered: "sub_labels not defined". I moved the sub_labels definition above the label-definition and now the automation works. I'm not allowed to submit changes here without forking, but these are my lines 838-839: sub_labels: "{{ trigger.payload_json['after']['data']['sub_labels'] }}" label: >-

Elder-HVAC-Man commented 4 months ago

Testing with HenningL's change. Only get alerts IF zone filter switch is OFF. Can't have any zones or object filters as well or automation does not fire. When it does fire, I get alerts on my iWatch, but no snaps or video.

Not sure if it helps, but error message from logs:

Logger: homeassistant.helpers.template Source: helpers/template.py:2613 First occurred: 10:00:43 AM (3 occurrences) Last logged: 10:00:43 AM

Template variable warning: 'new_snapshot' is undefined when rendering 'DEBUG (in loop): Send Notification: {{ custom_filter and not home and zone_filter and zone_multi_filter and state_true and (new_snapshot or presence_changed or zone_only_changed or entered_zones_changed ) }} Info: sublabel: {{sub_labels | join(', ')}}, image: "{{base_url}}/api/frigate{{client_id}}/notifications/{{events[0]}}/{{attachment}}" Title: {{title}} message: {{message}} iOS sound: {{'Critical' if critical else 'disabled by alert once' if alert_once else 'enabled'}}, Android Sound: {{'disabled by alert once' if alert_once else 'enabled'}}, iOS url: /api/frigate{{client_id}}/notifications/{{id}}/{{camera + '/clip.mp4' if video|length>0 and wait.trigger.payload_json['type'] == 'end' else attachment }} video: "{{video}}" critical: {{critical}}, Triggers: Presence Changed: {{presence_changed}}, Entered Zones Changed: {{'True' if event['before']['data']['zones'] != event['after']['data']['zones'] else 'False'}}{{' - Disabled' if not zone_only}}, Conditions: Filters: Zones: Zone Filter Enabled: {{zone_only}}, Multi Zone Enabled: {{zone_multi}}, Required Zones: {{input_zones}}, Last Zones: {{last_zones}}, Entered zones: {{enteredzones|list|length}} - {{enteredzones}}, TEST: {{'PASS' if not zone_only or (not zone_multi and zones|select('in', enteredzones)|list|length ) else 'PASS (Multi)' if zone_only and zone_multi and zone_multi_filter else 'FAIL (Multi)' if zone_only and zone_multi else 'FAIL' }}, Object Filter: Input: {{input_labels}}, TEST: {{'PASS' if not labels|length or object in labels else 'FAIL'}}, Presence entity (not home): Entity: {{presence_entity}}, TEST: {{'PASS' if not home else 'FAIL'}}, Time Filter: Disabled times: {{disable_times}}, TEST: {{'PASS' if now().hour not in disable_times else 'FAIL'}} State Filter: state filter toggle on: {{state_only}}, state filter entity: {{input_entity}}, required states: {{input_states}}, TEST: {{'PASS' if state_true else 'FAIL'}}, Custom Filter: {{'PASS' if custom_filter else 'FAIL'}},'

Trying to view clip on iPhone, I get this message: {"message":"Event not found", "success": false}

HenningL-Atea commented 4 months ago

There are some inconsistencies with the alerts;

Alert tekst says "Person['person'] detected on the yard camera", probably parsing of detected objects label doesn't work as intended ( {{ label }} translates to Person ['person'] )
I get alerts from one camera, but with snapshot from another camera, seems like snapshot is correct, but camera name in alert is wrong.
Links for clips and snapshots doesn't work ("Event not found").

SgtBatten commented 4 months ago

There are some inconsistencies with the alerts;

Keep these coming, i appreciate it. Not much time so will update when i can. This is valuable though.

HenningL-Atea commented 4 months ago

There might be some helpful hints in NickM-27's thread here: https://github.com/blakeblackshear/frigate/discussions/11554 For instance, Nick uses payload: alert, while the blueprint seems to be using payload: new

SgtBatten commented 4 months ago

Yes that one i;ve already determiend is useful, or at least should be user configuable.

Amadou91 commented 4 months ago

I am having a similar issue with the latest release where it shows Person['person'] detected on Front_Camera.

Additionally, when a notification for my front camera is triggered, It is also triggering the notification for my other two cameras. The snapshot that was taken for my Front camera is then shown on my other two cameras.


Frigate Notification DEBUG (in loop): Send Notification: False Info: sublabel: , image: "/api/frigate/notifications/1717421235.90517-pl55wn/thumbnail.jpg" Title: message: Person['person'] detected on the Front Door camera. iOS sound: enabled, Android Sound: enabled, iOS url: /api/frigate/notifications/1717421236.461684-7y0s3n/thumbnail.jpg video: "" critical: False, Triggers: Presence Changed: False, Entered Zones Changed: False - Disabled, Conditions: Filters: Zones: Zone Filter Enabled: False, Multi Zone Enabled: False, Required Zones: [], Last Zones: [], Entered zones: 0 - [], TEST: PASS, Object Filter: Input: , TEST: PASS, Presence entity (not home): Entity: , TEST: PASS, Time Filter: Disabled times: [], TEST: PASS State Filter: state filter toggle on: False, state filter entity: , required states: [], TEST: PASS, Custom Filter: PASS, triggered by automation FRONT  mqtt topic frigate/reviews
9:28:21 AM - 4 minutes ago

SgtBatten commented 4 months ago

yeah it doesnt currently check the camera guys

rroller commented 4 months ago

Tested with the tip of frigate dev and the latest version of SgtBatten/HA_blueprints. Can confirm the "Person['person']" issue and the camera check missing (Notifications labeled with the wrong camera). Everything else seems to work as expected.

SgtBatten commented 3 months ago

Moved to OP

Elder-HVAC-Man commented 3 months ago

Turning on filters and selecting person causes all of my cameras NOT to send a notification.

SgtBatten commented 3 months ago

Seems to only work for the automations configured with Object Filter. I need to select person for all of my cams

Interesting. I do not have it set to anything and its working

SgtBatten commented 3 months ago

Turning on filters and selecting person causes all of my cameras NOT to send a notification.

Fixed this

rroller commented 3 months ago

Pulled the latest changes. Working great so far

jacobrian commented 3 months ago

Running Frigate Notifications (0.12.0.4h) Frigate Beta: 0.14.0-beta3

Presence Filter not working correctly if using a custom defined Presence group within configuration.yaml. The group is configured correctly and reporting properly under states, and was previously working with Stable Frigate + Stable Blueprint. Presence Filter works correctly if I use a specific person entity, but no longer functions with a group entity.

Attaching Debug output while firing event:

Frigate Notification DEBUG: Info: fps: 5, frigate event id: 1719082026.884214-g5ilmm, Objects: Person, Sub Labels: Config: camera(formatted): garage(Garage), Base URL: https://{{baseurl_masked}}, critical: False, alert once: False, Update Thumbnails: True, Video: , Target: group (input/formatted): briser_devices/briser_devices, cooldown: 30s, initial delay: 0s, color: #03a9f4, sound: default, android_auto: True, Group: garage-frigate-notification, Channel: , Sticky: True, Title: , Message: Person detected on the Garage camera., Subtitle: , tap_action: https://{{baseurl_masked}}/api/frigate/notifications/1719082026.884214-g5ilmm/garage/clip.mp4, button 1 Text/URL/Icon: View Clip (https://{{baseurl_masked}}/api/frigate/notifications/1719082026.884214-g5ilmm/garage/clip.mp4) , button 2 Text/URL/Icon: View Snapshot (https://{{baseurl_masked}}/api/frigate/notifications/1719082026.884214-g5ilmm/snapshot.jpg) , button 3 Text/URL/Icon: Silence New Notifications (silence-garage) , icon: mdi:account-outline tv: False, tv_position: center, tv_size: large, tv_duration: 10, tv_transparency: 0%, tv_interrupt: False, Filters: Zones: zone filter toggle on: False, Multi-Zone toggle on: False, Required zones: [], Entered Zones: [], Zone Filter TEST: PASS, Required objects TEST: Input: ['person', 'cat'], TEST: FAIL presence entity (not home): Entity: group.somebody_home TEST: PASS, disabled times: [], State Filter: state filter toggle on: False, state filter entity: , required states: [], State Filter TEST: PASS, Custom Filter: True triggered by automation Notify - Garage Cam mqtt topic frigate/reviews

SgtBatten commented 3 months ago

Required objects TEST: Input: ['person', 'cat'], TEST: FAIL

I could easily have intorduced a bug but this one in particular says it failed due to not being a person or cat.

You may need to go through the traces (and even enable extra traces) to find the right one.

jacobrian commented 3 months ago

Interestingly, the notifications show up as "Person detected...". I've removed all object filters and the behaviour is the same. The moment I switch it from an individual device tracker to a group entity, I get notifications no matter what.

SgtBatten commented 3 months ago

The moment I switch it from an individual device tracker to a group entity, I get notifications no matter what.

I just added a group of two people who were both home and i did not get any notifications and the trace shows it correctly failing at that point due to the group being home.

jacobrian commented 2 months ago

Thanks mate, I created new automations and it works as expected. I suspect copying and modifying the YAML from old version to new version caused some conflict. Sorry to mis-report. Totally my fault. If anyone else experiences what I did, just create clean new automations.

T3chArmy commented 2 months ago

Curious what the complexity is to add select multiple for review severity so I could have one automation for alerts and detections?

lygris commented 1 month ago

im trying to filter for just an audio event and it isn't sending notifications for it but if i turn filters off person works just fine.

Edit, even without filters audio events dont seem to trigger the notification

SgtBatten commented 1 month ago

audio events dont seem to trigger the notification

I've not even begun to look into how to use them as triggers. if you have, please feel free to add to this PR so we can have both options

lygris commented 1 month ago

audio events dont seem to trigger the notification

I've not even begun to look into how to use them as triggers. if you have, please feel free to add to this PR so we can have both options

Turns out audio events dont post to the events mqtt topic.

SgtBatten commented 1 month ago

audio events dont post to the events mqtt topic.

We aren't using events any more. Do they go to the reviews topic?

f1d094 commented 1 month ago

@SgtBatten What's the status on this PR? I heard a comment that it is functional but there is a huge lag. We rely heavily on your blueprint (thanks! by the way) and 0.14.0 has completely broken our use-case. I get alerts 100x a day with nothing there. I understand this is because of the change between events/reviews...but I need to know if this will be fixed soon. I'm going to roll back to 0.13.2 for now because I simply cannot live my life with my watch and phone blowing up every 10 mins for a non-alert.

Please advise. Any feedback/update/ETA welcome.

SgtBatten commented 1 month ago

because of the change between events/reviews

Nothing changed with the way the events topic worked. This is simply a new option.

What's the status on this PR?

No idea. I have no time

f1d094 commented 1 month ago

Nothing changed with the way the events topic worked. This is simply a new option.

@SgtBatten Something has changed. After upgrading to 0.14.0 I was getting at least 4 alerts per hour with empty/404 on images and nothing showing on Frigate when I go to the server directly. On 0.13.2 I get zero empty alerts. There is always an image sent and I can always find it on the server.

I found this conversation which was what brought me here: https://github.com/blakeblackshear/frigate/discussions/11847

f1d094 commented 1 month ago

No idea. I have no time

Now that I'm back on 0.13.2 everything is working as expected again but I'm curious: Do you expect to return to this project/blueprint or should someone be looking to fork this and pick-up where you've left off? If it will be a couple months, then no big deal. If more like a year...then I should maybe look at forking and/or rolling my own. Thoughts?

SgtBatten commented 1 month ago

Yeah interesting, I'm aware of the phantom alerts conversation and from my experience it happened in the older version too.

It's not going to be a year. I'm still tweaking the main branch regularly. What I lack is feedback and input from others generally. This pull request is working well for my use can and has been for a long time. The delayed messages I haven't worked out and I was hoping someone would be able to see what's going on.

SgtBatten commented 1 month ago

Then there are the open questions in the opening comment on how best to proceed.

f1d094 commented 1 month ago

Understood. Thanks for all your efforts, by the way. Once I'm back on 0.14 I'd be happy to do some testing and feedback.

Unfortunately there are other things in 0.14.0 that I'm not happy with and kindof break my use case, so it will likely be a long while. Mostly it's just sluggish and too many clicks to do things on mobile, which is 99.99967% of my use...and I experienced a ~30% performance hit server-side too. I plan on writing everything up for the devs there but since I'm only one guy and everyone else in the universe seems happy as a clam, I doubt my complaints will get much traction.

SgtBatten commented 1 month ago

I have some UI gripes with it also. Namely that the camera names are not visible anywhere.

Because I have some 4k cameras which don't work universally on all devices and browsers due to he encoding I have those cameras in frigate twice. One at 4k and one at whatever h.264 resolution works. But when viewing the cams, I can't tell which is which until I try to view events or love view and it either works or it fails.

f1d094 commented 1 month ago

My main complaint is the focus on the timeline, the timeline, the timeline, the timeline. This is great when you are first setting up your system or if you have a false negative and need to review a bunch of cameras at once to see what happened that the AI didn't catch...but this isn't me. I have submitted almost 10,000 images to Frigate+ and have everything dialed in. All I want is:

Alerts when someone breaks my property line (this works awesome on 0.13.2, thanks to you)
Instantaneous birdseye/camera view on phone/watch so I can look when there is an event that needs paying attention to...or something goes 'bump' in the night and there isn't an alert
Immediate event -> frigate+ submission - I've already submitted thousands of images. All I want now is to immediately be able to flag a false positive. I shouldn't have to go 'find' the event...let alone half a dozen clicks to do it.
Autoplay. I HATE autoplay. Show me a still image I can zoom in on to see if i care...if so, I will CLICK on it and get a fullscreen image that is useful. Video in a tiny box that starts 5seconds before the "event" just wastes time.

Also, I think a lot of the sluggishness on mobile is down to whatever video they're forcing. My bookmarked birdseye page on 0.13.2 using "jsmpeg" loads 11 cameras instantaneously. The new setup? About 3-5 seconds. That's too long if someone is about to kick in your door or window.

I could go on...but I'm waaaayyy off topic so I'll just shush now.

maxoss commented 1 month ago

@SgtBatten just to check. What is the latest beta? On the main page beta is linked to 0.12.0.4e and in this pull requested I see 0.12.0.4h

SgtBatten commented 1 month ago

For frigate events based blueprints the main branch is the latest. This branch is purely for the new frigate 0.14 reviews topic. The version number is not really relevant here

SgtBatten commented 3 weeks ago

Anyone noticed the delay is gone? Mine are instant now.

Video previews are working in general, initially the video is short but updates to push and it gets longer.

Jonnehs commented 3 weeks ago

thanks for pointing me this way, i'd love to help test but I'm afraid I'm only a cut and paste developer (a kind way of saying, I'm not a developer) - I can get your blueprint automation to fire, but it stops on the "Notifications enabled for object label"

Notifications are enabled for people and cars, and a person triggered it but it seems to stop there

SgtBatten commented 3 weeks ago

but it stops on the "Notifications enabled for object label"

you dont appear to be using the version from this pull request. it should have three potential triggers in the first line, yours has 2.

Jonnehs commented 3 weeks ago

but it stops on the "Notifications enabled for object label"

you dont appear to be using the version from this pull request. it should have three potential triggers in the first line, yours has 2.

Ah yes, I just noticed and fixed this as you posted and it is firing now - however the animated GIF/android option doesn't seem to be working, the notifications just show the cropped snapshot, the video output to the notification is blank

Result:
params:
  domain: notify
  service: phones
  service_data:
    title: CCTV Alert - Front
    message: Car, Person detected - Front
    data:
      tag: 1725887456.695976-09pr1f
      group: front-frigate-notification
      color: '#926bc7'
      subject: ''
      image: >-
        https://URL.ui.nabu.casa/api/frigate/notifications/1725887456.695976-09pr1f/snapshot.jpg?bbox=1&crop=1&format=android
      video: ''
      clickAction: >-
        https://URL.ui.nabu.casa/api/frigate/notifications/1725887456.695976-09pr1f/front/clip.mp4
      ttl: 0
      priority: high
      notification_icon: mdi:cctv
      sticky: true
      channel: ''
      car_ui: true
      subtitle: ''
      fontsize: large
      position: center
      duration: 10
      transparency: 0%
      interrupt: false
      url: >-
        https://URL.ui.nabu.casa/api/frigate/notifications/1725887456.695976-09pr1f/front/clip.mp4
      attachment:
        url: >-
          https://URL.ui.nabu.casa/api/frigate/notifications/1725887456.695976-09pr1f/snapshot.jpg?bbox=1&crop=1
      push:
        sound:
          name: default
          volume: 1
          critical: 0
      entity_id: ''

SgtBatten commented 3 weeks ago

the video output to the notification is blank

is it set in the automation config?

please post the automation config

Jonnehs commented 3 weeks ago

the video output to the notification is blank

is it set in the automation config?

please post the automation config

yes it is

alias: Frigate CCTV - Front
description: ""
use_blueprint:
  path: SgtBatten/Beta.yaml
  input:
    camera: camera.front
    notify_device: bb2742edeb86557d75cfa8be96712079
    notify_group: notify.phones
    base_url: https://URL.ui.nabu.casa/
    mqtt_topic: frigate/reviews
    attachment: snapshot.jpg?bbox=1&crop=1
    video: >-
      {{base_url}}/api/frigate{{client_id}}/notifications/{{review_id}}/review_preview.gif?format=ts
    update_thumbnail: true
    message: "{{ label }} detected - {{ camera_name }}"
    title: CCTV Alert - Front
    color: "#926bc7"
    icon: mdi:cctv
    android_auto: true
    sticky: true
    zone_filter: true
    zones:
      - frontgarden
    labels:
      - person
      - car

SgtBatten commented 3 weeks ago

okay, i see an unintentional difference in the code for groups. it only sends the video at the end of the event. it's not appearing later?

[x] fix groups to send video always

Jonnehs commented 3 weeks ago

okay, i see an unintentional difference in the code for groups. it only sends the video at the end of the event. it's not appearing later?

No I left the notifications on sticky too see if they would eventually update, but no they never replace the snapshot image with an animated gif

SgtBatten commented 3 weeks ago

headed to bed but try 0.12.0.4j i just updated. failing that, try deleting the group and use a single mobile device.

Jonnehs commented 3 weeks ago

headed to bed but try 0.12.0.4j i just updated. failing that, try deleting the group and use a single mobile device.

Your update fixed it, still sending to the group.

The format is slightly strange now, the original snapshot image remains to the right of the notification after the GIF updates to the main image below, its a bit odd. If there is a way to change it I think I'd prefer to just have more screen space for the GIF.

Otherwise its working well, if a little slow. I will update the automations for my other cameras and give it a good test to see if there are any issues.

SgtBatten commented 3 weeks ago

Could be affected by device resolution.

On my pixel 7 pro it's full width and a smaller snapshot when expanded.

![Uploading Screenshot_20240910-073941.png…]()

Edit: Minimising and maximising it seems to resolve it when it appears like this. it seems to only happen if im alreayd looking at te notifcation (expanded) when it changes to a gif

JoshuaSeidel commented 1 week ago

For your thoughts section using frigate/reviews might help get rid of the multi notification on iOS. Every update with events comes through as a separate notification

SgtBatten commented 1 week ago

For your thoughts section using frigate/reviews might help get rid of the multi notification on iOS. Every update with events comes through as a separate notification

can you clarify what you are saying here. noone has reported separate notifications

JoshuaSeidel commented 1 week ago

For your thoughts section using frigate/reviews might help get rid of the multi notification on iOS. Every update with events comes through as a separate notification

can you clarify what you are saying here. noone has reported separate notifications

So when I go into my backyard. My backyard camera will detect me. Notification... then it will detect my face and update the event, or update the event with a new picture, which will trigger another notification, so I usually get at least 2 alerts for each time frigate detects something. Doesn't always happen but happens more than not

SgtBatten commented 1 week ago

which will trigger another notification

it should only be one notification. multiple updates to it sure depending on config, but only one actual thing to view on your phone

SgtBatten / HA_blueprints

Frigate 0.14 #216