blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.2k stars 1.76k forks source link

[Support]: Frigate Notification Snapshot #2527

Closed Daniel-dev22 closed 2 years ago

Daniel-dev22 commented 2 years ago

Describe the problem you are having

How do I use the snapshot from the "best image" in the events tab? The notifications are detection frame snapshots. I am only using one stream per camera as it's a blue iris stream I am pulling 1280x720 streams. The best image snapshots are of a much higher quality, have a bounding box and are not zoomed in.

I tried modifying the blueprint with bbox=1 and it didn't work.

  title: View Snapshot
  uri: '{{base_url}}/api/frigate/notifications/{{id}}/snapshot.jpg?bbox=1'

Version

0.9.4-26AE608

Frigate config file

mqtt:
  host: 192.168.4.61
  user: user
  password: pw
detectors:
  coral1:
    type: edgetpu
    device: usb:0
  coral2:
    type: edgetpu
    device: usb:1
cameras:
  dining_room_1:
    ffmpeg:
      input_args: 
        - -avoid_negative_ts
        - make_zero
        - -flags
        - low_delay
        - -strict
        - experimental 
        - -fflags
        - +genpts+discardcorrupt
        - -rtsp_transport
        - tcp
        - -stimeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      inputs:
        - path: rtsp://192.168.1.246:81/diningroom1?w=1280&h=720
          roles:
            - detect
            - rtmp
    detect:
      width: 1280
      height: 720
      fps: 5
    zones:
      dining_room_walkway:
        coordinates: 92,720,521,594,924,427,760,377,44,586
      dining_room_sitting_area:
        coordinates: 1280,720,1280,574,1025,398,222,720
      dining_room:
        coordinates: 82,566,0,286,370,197,631,391
  dining_room_2:
    ffmpeg:
      input_args: 
        - -avoid_negative_ts
        - make_zero
        - -flags
        - low_delay
        - -strict
        - experimental 
        - -fflags
        - +genpts+discardcorrupt
        - -rtsp_transport
        - tcp
        - -stimeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      inputs:
        - path: rtsp://192.168.1.246:81/diningroom2?w=1280&h=720
          roles:
            - detect
            - rtmp
    detect:
      width: 1280
      height: 720
      fps: 5
  outdoor_3:
    ffmpeg:
      input_args: 
        - -avoid_negative_ts
        - make_zero
        - -flags
        - low_delay
        - -strict
        - experimental 
        - -fflags
        - +genpts+discardcorrupt
        - -rtsp_transport
        - tcp
        - -stimeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      inputs:
        - path: rtsp://192.168.1.246:81/outdoor3?w=1280&h=720
          roles:
            - detect
            - rtmp
    detect:
      width: 1280
      height: 720
      fps: 5
    zones:
      drive_way:
        coordinates: 234,720,1280,720,1280,477,1280,425,958,305,202,276
  outdoor_4:
    ffmpeg:
      input_args: 
        - -avoid_negative_ts
        - make_zero
        - -flags
        - low_delay
        - -strict
        - experimental 
        - -fflags
        - +genpts+discardcorrupt
        - -rtsp_transport
        - tcp
        - -stimeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      inputs:
        - path: rtsp://192.168.1.246:81/outdoor4?w=1280&h=720
          roles:
            - detect
            - rtmp
    detect:
      width: 1280
      height: 720
      fps: 5
  outdoor_2:
    ffmpeg:
      input_args: 
        - -avoid_negative_ts
        - make_zero
        - -flags
        - low_delay
        - -strict
        - experimental 
        - -fflags
        - +genpts+discardcorrupt
        - -rtsp_transport
        - tcp
        - -stimeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      inputs:
        - path: rtsp://192.168.1.246:81/outdoor2?w=1280&h=720
          roles:
            - detect
            - rtmp
    detect:
      width: 1280
      height: 720
      fps: 5
    zones:
      front_door:
        coordinates: 913,430,1063,626,1136,577,986,416
      side:
        coordinates: 458,271,880,271,994,720,0,720,0,317
  outdoor_1:
    ffmpeg:
      input_args: 
        - -avoid_negative_ts
        - make_zero
        - -flags
        - low_delay
        - -strict
        - experimental 
        - -fflags
        - +genpts+discardcorrupt
        - -rtsp_transport
        - tcp
        - -stimeout
        - "5000000"
        - -use_wallclock_as_timestamps
        - "1"
      inputs:
        - path: rtsp://192.168.1.246:81/outdoor1?w=1280&h=720
          roles:
            - detect
            - rtmp
    detect:
      width: 1280
      height: 720
      fps: 5
objects:
  track:
    - person
    - car
snapshots:
  enabled: True

Relevant log output

N/a

FFprobe output from your camera

N/a

Frigate stats

No response

Operating system

HassOS

Install method

HassOS Addon

Coral version

USB

Network connection

Wired

Camera make and model

Blue Iris

Any other information that may be helpful

No response

Daniel-dev22 commented 2 years ago

This is the best image snapshot. Screenshot_20211225-062520_Home Assistant

blakeblackshear commented 2 years ago

it didn't work

I am not sure what you mean. That url changes what you see when you press the View Snapshot button in the notification. It doesn't change the image sent in the notification. Can you describe what you are trying to accomplish?

Daniel-dev22 commented 2 years ago

I'm trying to have the image I see before clicking on anything be that image above in the screenshot. Not the detection frame snapshot. Meaning the notification preview image I want it to be the image above.

Instead of the detection frame in this screenshot I'd like the "best image" snapshot from the events tab.

I just noticed that the bbox=1 works I didn't know it was only if you hit "view snapshot". The question is how can I get that same image to be the preview for the notification so I don't have to hit view snapshot to see the high quality image.

Screenshot_20211226-110244_Home Assistant Screenshot_20211225-062520_Home Assistant

blakeblackshear commented 2 years ago

Post your automation yaml

Daniel-dev22 commented 2 years ago
alias: Frigate Notification
description: ''
use_blueprint:
  path: hunterjm/frigate_notification.yaml
  input:
    camera: outdoor_4
    notify_device: 2d33724ba35dc14184cd04b0893101e2
    cooldown: 32
    base_url: https://link.ui.nabu.casa

I added the bbox=1 in the blueprint.yaml. I changed my real link above to "link" for privacy but it works as I said if I hit view snapshot.

It opens up chrome when I hit view snapshot and shows me this. Probably the same image as the one that's the "best image" in the events tab. I don't care for the extra network traffic to send the higher quality image by default. I previously was using a blue iris method of getting snapshots and sending high quality images.

Screenshot_20211226-111612_Chrome

blakeblackshear commented 2 years ago

That's the blueprint yaml. I need the full automation yaml generated from the blueprint to show you where to modify it.

Daniel-dev22 commented 2 years ago

Sorry here's the full blueprint.yaml

blueprint:
  name: Frigate Notification
  description: "## Frigate Mobile App Notification\n\nThis blueprint will send a notification\
    \ to your device when a Frigate event for the selected camera is fired. The notification\
    \ will initially include the thumbnail of the detection, but will update to include\
    \ actionable notifications allowing you to view the saved clip/snapshot when available,\
    \ or silence the notification for a configurable amount of time.\n\nWith this\
    \ blueprint, you may send the notification to multiple devices by leaving \"Device\"\
    \ blank and instead use a [notification group][1].\n\n### Required entities:\n\
    \  - Frigate Camera Name\n  - Mobile App Device **or** the name of a Notification\
    \ Group\n\n### Optional features:\n  - You can limit notifications to objects\
    \ entering **any** pre-defined [zones][2] in Frigate.\n  - You can specify which\
    \ [zones][2] to be notified about. This must be a list (e.g.):\n    ```yaml\n\
    \    - backyard\n    ```\n  - You can specify what type of [objects][3] to be\
    \ notified about. This must be a list (e.g.):\n    ```yaml\n    - person\n   \
    \ - car\n    ```\n  - You can disable notifications if a presence entity or group\
    \ is \"home\".\n  - You can configure a cooldown for the camera to reduce the\
    \ number of notifications when back-to-back events occur.\n  - You can silence\
    \ future notifications for a defined amount of time through actionable notifications.\
    \ This is helpful in situations where you know you will be triggering detections\
    \ for an extended period of time, like when the kids are playing outside.\n\n\
    [1]: https://companion.home-assistant.io/docs/notifications/notifications-basic#sending-notifications-to-multiple-devices\n\
    [2]: https://blakeblackshear.github.io/frigate/configuration/cameras#zones\n[3]:\
    \ https://blakeblackshear.github.io/frigate/configuration/objects\n"
  domain: automation
  source_url: https://gist.github.com/hunterjm/23c1588a9f2b8b9c2a62ffc364e17f8c
  input:
    camera:
      name: Frigate Camera
      description: The name of the camera as defined in your frigate configuration.
    notify_device:
      name: Device
      description: The device must run the official Home Assistant app to receive
        notifications.
      default: false
      selector:
        device:
          integration: mobile_app
    notify_group:
      name: Notification Group
      description: The name of the notification group to call.
      default: ''
    base_url:
      name: (Optional) Base URL
      description: 'The external url for your Home Assistant instance. This will default
        to a relative URL and will open the clips in the app instead of the browser,
        which may cause issues on some devices.

        '
      default: ''
    zone_filter:
      name: (Optional) Zone Filter
      description: Only notify if object has entered a defined zone.
      default: false
      selector:
        boolean: {}
    zones:
      name: (Optional) Trigger Zones
      description: A list (-) of zones you wish to recieve notifications for.
      default: []
      selector:
        object: {}
    labels:
      name: (Optional) Trigger Objects
      description: A list (-) of objects you wish to recieve notifications for.
      default: []
      selector:
        object: {}
    presence_filter:
      name: (Optional) Presence Filter
      description: Only notify if selected presence entity is not "home".
      default: ''
      selector:
        entity: {}
    cooldown:
      name: (Optional) Cooldown
      description: Delay before sending another notification for this camera after
        the last event.
      default: 30
      selector:
        number:
          max: 300.0
          min: 0.0
          unit_of_measurement: seconds
          mode: slider
          step: 1.0
    silence_timer:
      name: (Optional) Silence Notifications
      description: 'How long to silence notifications for this camera when requested
        as part of the actionable notification.

        '
      default: 30
      selector:
        number:
          max: 300.0
          min: 0.0
          unit_of_measurement: minutes
          mode: slider
          step: 1.0
mode: single
max_exceeded: silent
trigger:
  platform: mqtt
  topic: frigate/events
  payload: !input 'camera'
  value_template: '{{ value_json[''after''][''camera''] }}'
variables:
  id: '{{ trigger.payload_json[''after''][''id''] }}'
  camera: '{{ trigger.payload_json[''after''][''camera''] }}'
  camera_name: '{{ camera | replace(''_'', '' '') | title }}'
  object: '{{ trigger.payload_json[''after''][''label''] }}'
  label: '{{ object | title }}'
  entered_zones: '{{ trigger.payload_json[''after''][''entered_zones''] }}'
  type: '{{ trigger.payload_json[''type''] }}'
  base_url: !input 'base_url'
  group_target: !input 'notify_group'
  zone_only: !input 'zone_filter'
  input_zones: !input 'zones'
  zones: '{{ input_zones | list }}'
  input_labels: !input 'labels'
  labels: '{{ input_labels | list }}'
  presence_entity: !input 'presence_filter'
condition:
- '{{ type != ''end'' }}'
- '{{ not zone_only or entered_zones|length > 0 }}'
- '{{ not zones|length or zones|select(''in'', entered_zones)|list|length > 0 }}'
- '{{ not labels|length or object in labels }}'
- '{{ not presence_entity or not is_state(presence_entity, ''home'') }}'
action:
- choose:
  - conditions: '{{ not group_target }}'
    sequence:
    - device_id: !input 'notify_device'
      domain: mobile_app
      type: notify
      message: A {{ label }} was detected on the {{ camera_name }} camera.
      data:
        tag: '{{ id }}'
        group: frigate-notification-{{ camera }}
        image: /api/frigate/notifications/{{id}}/thumbnail.jpg?format=android
        attachment:
          url: /api/frigate/notifications/{{id}}/thumbnail.jpg
  default:
  - service: notify.{{ group_target }}
    data:
      message: A {{ label }} was detected on the {{ camera_name }} camera.
      data:
        tag: '{{ id }}'
        group: frigate-notification-{{ camera }}
        image: /api/frigate/notifications/{{id}}/thumbnail.jpg?format=android
        attachment:
          url: /api/frigate/notifications/{{id}}/thumbnail.jpg
- repeat:
    sequence:
    - wait_for_trigger:
      - platform: mqtt
        topic: frigate/events
        payload: '{{ id }}'
        value_template: '{{ value_json[''after''][''id''] }}'
      timeout:
        minutes: 2
      continue_on_timeout: false
    - condition: template
      value_template: '{{ wait.trigger.payload_json[''type''] == ''end'' }}'
    - choose:
      - conditions: '{{ not group_target }}'
        sequence:
        - device_id: !input 'notify_device'
          domain: mobile_app
          type: notify
          message: A {{ label }} was detected on the {{ camera_name }} camera.
          data:
            tag: '{{ id }}'
            group: frigate-notification-{{ camera }}
            url: '{{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4'
            clickAction: '{{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4'
            image: /api/frigate/notifications/{{id}}/thumbnail.jpg?format=android
            sound: none
            attachment:
              url: /api/frigate/notifications/{{id}}/thumbnail.jpg
            actions:
            - action: URI
              title: View Clip
              uri: '{{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4'
            - action: URI
              title: View Snapshot
              uri: '{{base_url}}/api/frigate/notifications/{{id}}/snapshot.jpg?bbox=1'
            - action: silence-{{ camera }}
              title: Silence Notifications
              destructive: true
      default:
      - service: notify.{{ group_target }}
        data:
          message: A {{ label }} was detected on the {{ camera_name }} camera.
          data:
            tag: '{{ id }}'
            group: frigate-notification-{{ camera }}
            url: '{{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4'
            clickAction: '{{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4'
            image: /api/frigate/notifications/{{id}}/thumbnail.jpg?format=android
            sound: none
            attachment:
              url: /api/frigate/notifications/{{id}}/thumbnail.jpg
            actions:
            - action: URI
              title: View Clip
              uri: '{{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4'
            - action: URI
              title: View Snapshot
              uri: '{{base_url}}/api/frigate/notifications/{{id}}/snapshot.jpg?bbox=1'
            - action: silence-{{ camera }}
              title: Silence Notifications
              destructive: true
    until: '{{ wait.trigger.payload_json[''type''] == ''end'' }}'
- wait_for_trigger:
  - platform: event
    event_type: mobile_app_notification_action
    event_data:
      action: silence-{{ camera }}
  timeout:
    seconds: !input 'cooldown'
  continue_on_timeout: false
- delay:
    minutes: !input 'silence_timer'
blakeblackshear commented 2 years ago

I was talking about the yaml for the automation created from the blueprint. You can see it if you go into automations in the config section, select your automation, select "Show Trace" > "Automation config". You will need to copy that out into your own custom automation and change all of the thumbnail.jpg references to snapshot.jpg references. You can also remove the ?format=android parameter.

Daniel-dev22 commented 2 years ago
mode: single
max_exceeded: silent
trigger:
  platform: mqtt
  topic: frigate/events
  payload: outdoor_4
  value_template: '{{ value_json[''after''][''camera''] }}'
variables:
  id: '{{ trigger.payload_json[''after''][''id''] }}'
  camera: '{{ trigger.payload_json[''after''][''camera''] }}'
  camera_name: '{{ camera | replace(''_'', '' '') | title }}'
  object: '{{ trigger.payload_json[''after''][''label''] }}'
  label: '{{ object | title }}'
  entered_zones: '{{ trigger.payload_json[''after''][''entered_zones''] }}'
  type: '{{ trigger.payload_json[''type''] }}'
  base_url: https://link.ui.nabu.casa
  group_target: ''
  zone_only: false
  input_zones: []
  zones: '{{ input_zones | list }}'
  input_labels: []
  labels: '{{ input_labels | list }}'
  presence_entity: ''
condition:
  - '{{ type != ''end'' }}'
  - '{{ not zone_only or entered_zones|length > 0 }}'
  - '{{ not zones|length or zones|select(''in'', entered_zones)|list|length > 0 }}'
  - '{{ not labels|length or object in labels }}'
  - '{{ not presence_entity or not is_state(presence_entity, ''home'') }}'
action:
  - choose:
      - conditions: '{{ not group_target }}'
        sequence:
          - device_id: 2d33724ba35dc14184cd04b0893101e2
            domain: mobile_app
            type: notify
            message: A {{ label }} was detected on the {{ camera_name }} camera.
            data:
              tag: '{{ id }}'
              group: frigate-notification-{{ camera }}
              image: /api/frigate/notifications/{{id}}/thumbnail.jpg?format=android
              attachment:
                url: /api/frigate/notifications/{{id}}/thumbnail.jpg
    default:
      - service: notify.{{ group_target }}
        data:
          message: A {{ label }} was detected on the {{ camera_name }} camera.
          data:
            tag: '{{ id }}'
            group: frigate-notification-{{ camera }}
            image: /api/frigate/notifications/{{id}}/thumbnail.jpg?format=android
            attachment:
              url: /api/frigate/notifications/{{id}}/thumbnail.jpg
  - repeat:
      sequence:
        - wait_for_trigger:
            - platform: mqtt
              topic: frigate/events
              payload: '{{ id }}'
              value_template: '{{ value_json[''after''][''id''] }}'
          timeout:
            minutes: 2
          continue_on_timeout: false
        - condition: template
          value_template: '{{ wait.trigger.payload_json[''type''] == ''end'' }}'
        - choose:
            - conditions: '{{ not group_target }}'
              sequence:
                - device_id: 2d33724ba35dc14184cd04b0893101e2
                  domain: mobile_app
                  type: notify
                  message: A {{ label }} was detected on the {{ camera_name }} camera.
                  data:
                    tag: '{{ id }}'
                    group: frigate-notification-{{ camera }}
                    url: >-
                      {{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4
                    clickAction: >-
                      {{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4
                    image: >-
                      /api/frigate/notifications/{{id}}/thumbnail.jpg?format=android
                    sound: none
                    attachment:
                      url: /api/frigate/notifications/{{id}}/thumbnail.jpg
                    actions:
                      - action: URI
                        title: View Clip
                        uri: >-
                          {{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4
                      - action: URI
                        title: View Snapshot
                        uri: >-
                        {{base_url}}/api/frigate/notifications/{{id}}/snapshot.jpg?bbox=1
                      - action: silence-{{ camera }}
                        title: Silence Notifications
                        destructive: true
          default:
            - service: notify.{{ group_target }}
              data:
                message: A {{ label }} was detected on the {{ camera_name }} camera.
                data:
                  tag: '{{ id }}'
                  group: frigate-notification-{{ camera }}
                  url: >-
                    {{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4
                  clickAction: >-
                    {{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4
                  image: >-
                    /api/frigate/notifications/{{id}}/thumbnail.jpg?format=android
                  sound: none
                  attachment:
                    url: /api/frigate/notifications/{{id}}/thumbnail.jpg
                  actions:
                    - action: URI
                      title: View Clip
                      uri: >-
                        {{base_url}}/api/frigate/notifications/{{id}}/{{camera}}/clip.mp4
                    - action: URI
                      title: View Snapshot
                      uri: >-
                        {{base_url}}/api/frigate/notifications/{{id}}/snapshot.jpg?bbox=1
                    - action: silence-{{ camera }}
                      title: Silence Notifications
                      destructive: true
      until: '{{ wait.trigger.payload_json[''type''] == ''end'' }}'
  - wait_for_trigger:
      - platform: event
        event_type: mobile_app_notification_action
        event_data:
          action: silence-{{ camera }}
    timeout:
      seconds: 32
    continue_on_timeout: false
  - delay:
      minutes: 30
id: '1640340866850'
alias: Frigate Notification
description: ''
blakeblackshear commented 2 years ago

That's it. You will need to modify that yaml as I said above and copy it into your automations.yaml.

Daniel-dev22 commented 2 years ago

That's it. You will need to modify that yaml as I said above and copy it into your automations.yaml.

Sounds good thank you for the help.

Daniel-dev22 commented 2 years ago

I may have closed it too soon lol. When I paste the Automation as it is above it says message malformed: required key not provided @data ['action']

blakeblackshear commented 2 years ago

You have to put it as a list item like the other automations.

Daniel-dev22 commented 2 years ago

Not sure I follow. What do you mean by "list item"

blakeblackshear commented 2 years ago

The yaml generated by the blueprint is just another automation you can include in your automations.yaml like this: https://www.home-assistant.io/docs/automation/yaml/#migrating-your-yaml-automations-to-automationsyaml

Daniel-dev22 commented 2 years ago

I tried including It using an Id and it didn't work. I ended up just modifying the blueprint yaml and saving a copy in the event its updated I can just replace the blueprint.yaml again for it to work.

Is there a way for the notification to overwrite itself. Meaning instead of 50 notifications for 1 camera it's just 1 that constantly overwrites itself.

I used this on a blue iris notification for that but I'm not sure where I would add this

tag: Outdoor 3
persistent: true
sticky: true
channel: Outdoor 3
blakeblackshear commented 2 years ago

You will need to modify the blueprint again for each call to the notify service. You can find where those values go here: https://companion.home-assistant.io/docs/notifications/notifications-basic/

Daniel-dev22 commented 2 years ago

Sounds good I figured that was what I had to do.