Developing a Notifier component, need guidance

First of all thanks for the code! It was a relative breeze setting up Viseron and the devContainer stuff works (almost) like a charm!

First, one thing about setting up Viseron with cuda support on Ubuntu. The requirements state to install a deprecated package at https://viseron.netlify.app/docs/documentation/installation. The replacement package also works, but, on Ubuntu, after installing the NVIDIA Container Toolkit, I did have to manually add the runtime to /etc/docker/daemon.json, like so:

{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

and then reload that and restart docker service like so:

sudo systemctl daemon-reload
sudo systemctl restart docker

After that, configuring and running Viseron with a darknet object detector on GPU worked like a charm!

Before I continue with my proposal for a Notifier component, one thing about setting up the development environment. When following the instructions outlined at https://viseron.netlify.app/docs/developers/development_environment/setup my devContainer would fail during initialization and dump me into a debug session for the container from where it said that this mount's type is invalid or that there is no directory on the host:

https://github.com/roflcoopter/viseron/blob/8ee81a49f137d0b6d64a952b628b14a997cf09c2/.devcontainer/devcontainer.json#L22

After removing that line the devContainer does initialize and I'm able to develop and debug the code. That's frickin' amazing!

Now, on to the meat. I wanted to get notifications of objects being detected, so I've made a first rudimentary version of a Notifier component. The Notifier component can send messages through a Telegram bot or via email, or both. My first question: I'm refactoring it now and would like to get your input on the following suggestion for a config for the Notifier.

notifier:
  cameras:
    frontdoorcam:
      detection_labels: ['person'] # trigger notifier(s) on these detected objects
      smtp:
        smtp_server: mail.yourhost.com
        smtp_password: abc
        smtp_username: you@yourhost.com
        smtp_sender: you@yourhost.com
        smtp_recipients: someone@somewhere.com
        smtp_port: 587
      telegram:
        telegram_bot_token: bot_token_generated_by_bot_father
        telegram_chat_ids: [1,2,3] # chat ids to send the notifications to
        send_thumbnail: true # send a photo of the detected object
        send_video: true # send the video after it's finished recording
        accept_new_chat_ids: false # whether the Telegram Bot should listen for updates and store new chat ids to which to send notifs
        chat_join_password: define_your_password_here # joining the Bot (by sending "/join") will prompt for this password

Second question: the current implementation, which does not use the above config, but a simpler version, uses the state change listener:

self._vis.listen_event(EVENT_STATE_CHANGED, self.state_changed)

to listen for the recording state of the camera to switch to 'off' after a person has been detected by it

def state_changed(self, event_data: Event) -> None:
        """On event do notify."""
        LOGGER.info(event_data)
        if (
            event_data.data.entity_id.startswith("binary_sensor.")
            and event_data.data.entity_id.endswith("_recorder")
            and event_data.data.current_state
            and event_data.data.current_state.state == "off"
            and self._object_was_detected
        ):
            self._object_was_detected = False
            self.notify_email(event_data)
            asyncio.run(self.notify_telegram(event_data))
        elif (
            event_data.data.entity_id.endswith("_object_detected_person")
            and event_data.data.current_state
            and event_data.data.current_state.state == "on"
        ):
            self._object_was_detected = True

and then sends a(n) (email and) telegram message:

async def notify_telegram(self, event_data) -> None:
        """Send notification."""
        bot_token = self._config[CONFIG_TELEGRAM_BOT_TOKEN]
        chat_id = self._config[CONFIG_TELEGRAM_CHAT_ID]
        bot = Bot(token=bot_token)
        await bot.send_photo(
            chat_id=chat_id,
            photo=open(
                event_data.data.previous_state.attributes["thumbnail_path"], "rb"
            ),
            caption=f"Camera: {event_data.data.entity_id} detected a {event_data.data.previous_state.attributes['objects'][0].formatted['label']} with score {event_data.data.previous_state.attributes['objects'][0].formatted['confidence']}",
        )

Where I'm currently hitting a snag is when I try to send the video associated with the recording. Though the event_data contains a path to the recording, opening it fails (because it does not exist, yet?):

await bot.send_video(
             chat_id=chat_id,
             video=open(event_data.data.previous_state.attributes["path"], "rb"),
             caption=f"Camera: {event_data.data.entity_id} detected a {event_data.data.previous_state.attributes['objects'][0].formatted['label']} with score {event_data.data.previous_state.attributes['objects'][0].formatted['confidence']}",
         )

Might this have something to do with the Tiered storage changes?

Again, thanks for an amazing product and let me know what you think about this Notifier component idea :)

First of all thanks for the code! It was a relative breeze setting up Viseron and the devContainer stuff works (almost) like a charm!

First, one thing about setting up Viseron with cuda support on Ubuntu. The requirements state to install a deprecated package at https://viseron.netlify.app/docs/documentation/installation. The replacement package also works, but, on Ubuntu, after installing the NVIDIA Container Toolkit, I did have to manually add the runtime to /etc/docker/daemon.json, like so:
{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}
and then reload that and restart docker service like so:
sudo systemctl daemon-reload
sudo systemctl restart docker
After that, configuring and running Viseron with a darknet object detector on GPU worked like a charm!

Before I continue with my proposal for a Notifier component, one thing about setting up the development environment. When following the instructions outlined at https://viseron.netlify.app/docs/developers/development_environment/setup my devContainer would fail during initialization and dump me into a debug session for the container from where it said that this mount's type is invalid or that there is no directory on the host:

https://github.com/roflcoopter/viseron/blob/8ee81a49f137d0b6d64a952b628b14a997cf09c2/.devcontainer/devcontainer.json#L22

After removing that line the devContainer does initialize and I'm able to develop and debug the code. That's frickin' amazing!

Thanks for the feedback on the Devcontainer! The mount is there to allow for saving the config in the repo and then mounting it to /config so that Viseron sees it in the devcontainer. Will see if i can make that work better.

notifier:
  cameras:
    frontdoorcam:
      detection_labels: ['person'] # trigger notifier(s) on these detected objects
      smtp:
        smtp_server: mail.yourhost.com
        smtp_password: abc
        smtp_username: you@yourhost.com
        smtp_sender: you@yourhost.com
        smtp_recipients: someone@somewhere.com
        smtp_port: 587
      telegram:
        telegram_bot_token: bot_token_generated_by_bot_father
        telegram_chat_ids: [1,2,3] # chat ids to send the notifications to
        send_thumbnail: true # send a photo of the detected object
        send_video: true # send the video after it's finished recording
        accept_new_chat_ids: false # whether the Telegram Bot should listen for updates and store new chat ids to which to send notifs
        chat_join_password: define_your_password_here # joining the Bot (by sending "/join") will prompt for this password

Second question: the current implementation, which does not use the above config, but a simpler version, uses the state change listener:

self._vis.listen_event(EVENT_STATE_CHANGED, self.state_changed)

to listen for the recording state of the camera to switch to 'off' after a person has been detected by it

def state_changed(self, event_data: Event) -> None:
        """On event do notify."""
        LOGGER.info(event_data)
        if (
            event_data.data.entity_id.startswith("binary_sensor.")
            and event_data.data.entity_id.endswith("_recorder")
            and event_data.data.current_state
            and event_data.data.current_state.state == "off"
            and self._object_was_detected
        ):
            self._object_was_detected = False
            self.notify_email(event_data)
            asyncio.run(self.notify_telegram(event_data))
        elif (
            event_data.data.entity_id.endswith("_object_detected_person")
            and event_data.data.current_state
            and event_data.data.current_state.state == "on"
        ):
            self._object_was_detected = True

and then sends a(n) (email and) telegram message:

async def notify_telegram(self, event_data) -> None:
        """Send notification."""
        bot_token = self._config[CONFIG_TELEGRAM_BOT_TOKEN]
        chat_id = self._config[CONFIG_TELEGRAM_CHAT_ID]
        bot = Bot(token=bot_token)
        await bot.send_photo(
            chat_id=chat_id,
            photo=open(
                event_data.data.previous_state.attributes["thumbnail_path"], "rb"
            ),
            caption=f"Camera: {event_data.data.entity_id} detected a {event_data.data.previous_state.attributes['objects'][0].formatted['label']} with score {event_data.data.previous_state.attributes['objects'][0].formatted['confidence']}",
        )

await bot.send_video(
             chat_id=chat_id,
             video=open(event_data.data.previous_state.attributes["path"], "rb"),
             caption=f"Camera: {event_data.data.entity_id} detected a {event_data.data.previous_state.attributes['objects'][0].formatted['label']} with score {event_data.data.previous_state.attributes['objects'][0].formatted['confidence']}",
         )

Might this have something to do with the Tiered storage changes?

Again, thanks for an amazing product and let me know what you think about this Notifier component idea :)

A notifier component sounds awesome! It has been requested a few times but i have never gotten around to it. I have had plans to implement https://github.com/caronc/apprise which provides many different kinds of notifiers, including Telegram Config format looks okay, i think it would be better to move the smtp config for instance to the same level as cameras so that you dont have to repeat it if you have multiple cameras for instance.

Regarding the video, yes you are correct that it does not exist when the recording finishes since it has to be concatenated. And in the current dev build with tiered storage there wont be any video created by default, since the frontend works by loading the small segments as an HLS playlist instead of full videos. You can however create full mp4 recordings the old way using the config option create_event_clip We could also concatenate a video on the fly if needed for a notification.

I saw you opened #787, with a lot of other questions, do you want to continue this discussion there?

roflcoopter / viseron

Developing a Notifier component, need guidance #785