search

djdd87 / SynoAI

A Synology Surveillance Station notification system utilising DeepStack AI
GNU General Public License v3.0
203 stars 24 forks source link

readme

UPDATE 2023-08-29

I've begun working on a new version of SynoAI from the ground up which has an administrative interface instead of a config based approach. I will also be taking the opportunity to implement a number of requested features that have been requested for a long time.

SynoAI

A Synology Surveillance Station notification system utilising DeepStack AI, inspired by Christopher Adams' sssAI implementation.

The aim of the solution is to reduce the noise generated by Synology Surveillance Station's motion detection by routing all motion events via a Deepstack docker image to look for particular objects, e.g. people.

While sssAI is a great solution, it is hamstrung by the Synology notification system to send motion alerts. Due to the delay between fetching the snapshot, processing the image using the AI and requesting the alert, it means that the image attached to the Synology notification is sometimes 5-10 seconds after the motion alert was originally triggered.

SynoAI aims to solve this problem by side-stepping the Synology notifications entirely by allowing other notification systems to be used.

Building LatestImage

Buy Me A Coffee! :coffee:

I made this application mostly for myself in order to improve upon Christopher Adams' original idea and don't expect anything in return. However, if you find it useful and would like to buy me a coffee, feel free to do it at Buy me a coffee! :coffee:. This is entirely optional, but would be appreciated! Or even better, help supported this project by contributing changes such as expanding the supported notification systems (or even AIs).

Versioning

The documentation you see here corresponds to the branch/tag selected above. The documentation will be accurate for the version tag you have selected, however if viewing the main branch, this is assumed to be the documentation that corresponds to the latest commit and latest image.

For example, if you are using the docker image/version v1.1.0, then ensure you have selected the tag for v1.1.0, otherwise you may see features or options which are not available on your version of SynoAI.

Table of Contents

Features

Config

An example appsettings.json configuration file can be found here and all configuration for notifications and AI can be found under their respective sections. The following are the top level configs for communication with Synology Surveillance Station:

General Config

Camera Config

Camera API

The following settings can be applied to the camera while SynoAI is running. All the following settings can be applied by performing a JSON POST to the respective Camera endpoint. Only the values that you wish to set need to be provided in the JSON body.

As an example, to disable the camera with the name Driveway on the SynoAI URL 10.0.0.10 port 8080, you POST to http://10.0.0.10:8080/Camera/Driveway with the following JSON body:

{
  "Enabled": false
}

To re-enable the camera, just POST again with Enabled set to true.

{
  "Enabled": true
}

Development Config

Configs which should be changed for debugging (change at own risk):

Supported AIs

In order to specify the AI to use, set the Type property against the AI section in the config:

"AI": {
  "Type": "DeepStack"
}

Deepstack

The Deepstack API is a free to use AI that can identify objects, faces and more. Currently SynoAI makes use of the object detection model, which allows detection of people, cars, bicycles, trucks and even giraffes! For a full list of supported types see the Deepstack documentation.

"AI": {
  "Type": "DeepStack",
  "Url": "http://10.0.0.10:83"
}

CodeProject.AI-Server

For a full list of supported types see the CodeProject.AI documentation.

"AI": {
  "Type": "CodeProjectAIServer",
  "Url": "http://10.0.0.10:83"
}

Notifications

Multiple notifications can be triggered when an object is detected. Each notification will have a defined "Type" and the sections below explain how each notification should be defined. An optional feature allows notifications to be triggered only by specified cameras.

"Notifiers": [
  {
    "Type": "{Type}",
    "Cameras": [ "Driveway", "Garden"]
  }
]

Pushbullet

The Pushbullet notification will send an image and a message containing a list of detected object types. An API key will need to be obtained from your Pushbullet account. Currently the notification will be sent to all devices that the API key belongs to.

{
  "Type": "Pushbullet",
  "ApiKey": "0.123456789"
}

Webhook

The webhook notification can be used to make web requests (e.g. API calls) either with or without the image that triggered the alert.

Configuration

{
  "Type": "Webhook",
  "Url": "http://servername/resource",
  "Method": "POST",
  "Authentication": "Bearer",
  "Token": "XYZ.123456",
  "SendImage": false,
  "AllowInsecureUrl": false
}

Example POST data

The following is example data for when SendImage is false and SynoAIUrl is "http://192.168.1.2".

{
  "camera": "Driveway",
  "foundTypes": [
    "Car"
  ],
  "predictions": [
    {
      "Label": "car",
      "Confidence": 67.89117,
      "MinX": 1738,
      "MinY": 420,
      "MaxX": 2304,
      "MaxY": 844,
      "SizeX": 566,
      "SizeY": 424
    }
  ],
  "message": "Motion detected on Driveway\n\nDetected 1 objects:\nCar",
  "imageUrl": "http://192.168.1.2/CameraName/capture.jpeg"
}

Telegram

The telegram bot will send notifications and images when motion has been detected. To use this notification, you will need to set up your own Telegram bot using one of the many guides available.

{
  "Type": "Telegram",
  "ChatID": "000000000",
  "Token": "",
  "PhotoBaseURL": ""
}

Email

The email notification will send and email with the attached image to the specified recipient.

{
  "Type": "Email",
  "Sender": "youremail@example.com",
  "Destination": "youremail@example.com",
  "Host": "smtp.server.com",
  "Port": 465,
  "Username": "username",
  "Password": "password",
  "Encryption": "Auto"
}

Gmail

Note that to send email using GMail you may need to log into your Google Account (or Admin Console) and allow "Less secure app access".

{
  "Type": "Email",
  "Sender": "youremail@gmail.com",
  "Destination": "youremail@gmail.com",
  "Host": "smtp.gmail.com",
  "Port": 587,
  "Encryption": "StartTLS",
  "Username": "youremail@gmail.com",
  "Password": "yourpassword"
},

HomeAssistant

Integration with HomeAssistant can be achieved using the Push integration and by calling the HomeAssistant webhook with the SynoAI Webhook notification.

Example HomeAssistant Configuration.yaml:

camera:
  - platform: push
    name: Motion Driveway
    webhook_id: motion_driveway
    timeout: 1
    buffer: 1

HomeAssistant requires the POSTed image field to be called "image" (which is the default for SynoAI), so from the SynoAI side the integration is simply a case of creating a Webhook and pointing it at your HomeAssistant's IP and Webhook ID:

{
  "Type": "Webhook",
  "Url": "http://nas-ip:8123/api/webhook/motion_driveway"
}

Automations can be created using this webhook by checking for changes for the camera entity state. When the Push camera is not receiving any data, it will be in the state of idle. When the state switches to recording, it means that the webhook has started receiving data. For the fastest automation responses, perform your actions immediately on that state change.

Multiple webhooks can be set up, each pointed at a different HomeAssistant Push camera. Additionally, you can create an automation that is triggered on a Webhook call. Then just use the SynoAI webhook notification to call that webhook. Note that it's wasteful to send an image when triggering the non-Push webhooks on HomeAssistant, so ensure that SendImage is set to false.

Pushover

The Pushover notification will send an image and a message containing a list of detected object types. An API key and user key will need to be obtained from your Pushover account. An array of devices can be specified to limit the devices that receive the notifications, or the device field can be left blank

{
  "Type": "Pushover",
  "ApiKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "UserKey": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "Device": [
    "iphone"
  ]
}

Discord

Send notifications to a Discord server via Discord Webhooks. You can get a Discord Webhook URL for your Discord server by following the instructions here.

{
  "Type": "Discord",
  "Url": " https://discord.com/api/webhooks/F4K3W3BH00K"
}

MQTT

Send notifications as MQTT messages. Messages are JSON-encoded and sent to the {BaseTopic}\{CameraName}\notification topic. You can optionally include the capture as a base64 encoded JPG, but this is disabled by default.

Configuration

{
  "Type": "MQTT",
  "Host": "mqtt.domain.com",
  "Port": 1883,
  "Username": "user",
  "Password": "password",
  "BaseTopic": "synoai",
  "SendImage": "false"
}

Example payload data

The following is example data for when SendImage is false and SynoAIUrl is "https://synoai.example.com".

{
  "camera": "Driveway",
  "foundTypes": [
    "Car"
  ],
  "predictions": [
    {
      "Label": "car",
      "Confidence": 67.89117,
      "MinX": 1738,
      "MinY": 420,
      "MaxX": 2304,
      "MaxY": 844,
      "SizeX": 566,
      "SizeY": 424
    }
  ],
  "message": "Motion detected on Driveway\n\nDetected 1 objects:\nCar",
  "imageUrl": "https://synoai.example.com/CameraName/capture.jpeg"
}

Caveats

Configuration

The configuration instructions below are primarily aimed at running SynoAI in a docker container on DSM (Synology's operating system). Docker will be required anyway as Deepstack is assumed to be setup inside a Docker container. It is entirely possible to run SynoAI on a webserver instead, or to install it on a Docker instance that's not running on your Synology NAS, however that is outside the scope of these instructions. Additionally, the configuration of the third party notification systems (e.g. generating a Pushbullet API Key) is outside the scope of these instructions and can be found on the respective applications help guides.

The top level steps are:

1) Configure Deepstack

The following instructions explain how to set up the Deepstack image using the Docker app built into DSM.

2) Configure SynoAI

The following instructions explain how to set up the SynoAI image using the Docker app built into DSM. For docker-compose, see the example file in the src, or in the documentation below.

3) Create Action Rules

The next step is to configure actions inside Surveillance Station that will call the SynoAI API.

Summary

Congratulations, you should now have a trigger calling the SynoAI API for your camera every time Surveillance Station detects motion. In order to set up multiple cameras, just create a new Action Rule for each camera.

Note that SynoAI is still reliant on Surveillance Station detecting the motion, so this will need some tuning on your part. However, it's now possible to up the sensitivity and avoid false-positives as SynoAI will only notify you (via your preferred notification system/app) when an object is detected, e.g. a Person.

Updating

If you have followed the above instructions to setup the container using the DSM UI, then the following instructions are how to correctly update:

Docker

SynoAI can be installed as a docker image, which is available from DockerHub.

Images

The following is a list of the available images and their meaning:

Docker Configuration

The image can be pulled using the Docker cli by calling:

docker pull djdd87/synoai:latest

To run the image a volume must be specified to map your appsettings.json file. Additionally a port needs mapping to port 80 in order to trigger the API. Optionally, the Captures directory can also be mapped to easily expose all the images output from SynoAI.

docker run 
  -v /path/appsettings.json:/app/appsettings.json 
  -v /path/captures:/app/Captures 
  -p 8080:80 djdd87/synoai:latest

Docker-Compose

version: '3.4'

services:
  synoai:
    image: djdd87/synoai:latest
    build:
      context: .
      dockerfile: ./Dockerfile
    ports:
      - "8080:80"
    volumes:
      - /docker/synoai/captures:/app/Captures
      - /docker/synoai/appsettings.json:/app/appsettings.json

Example appsettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Warning"
    }
  },

  "Url": "http://10.0.0.10:5000",
  "User": "SynologyUser",
  "Password": "SynologyPassword",

  "MinSizeX": 100,
  "MinSizeY": 100,

  "DaysToKeepCaptures": 14,

  "AI": {
    "Type": "DeepStack",
    "Url": "http://10.0.0.10:83"
  },

  "Notifiers": [
    {
      "Type": "Pushbullet",
      "ApiKey": "0.123456789"
    },
    {
      "Type": "Webhook",
      "Url": "http://server/images",
      "Method": "POST",
      "Field": "image"
    }
  ],

  "Cameras": [
    {
      "Name": "Driveway",
      "Types": [ "Person", "Car" ],
      "Threshold": 45,
      "MinSizeX": 250,
      "MinSizeY": 500
    },
    {
      "Name": "SideGate",
      "Types": [ "Person" ],
      "Wait": 2500
    },
    {
      "Name": "BackDoor",
      "Types": [ "Person" ],
      "Threshold": 30,
      "Exclusions": [
        {
            "Start": { "X": 1800, "Y": 400 },
            "End": { "X": 2350, "Y": 900 }
        },
        {
            "Start": { "X": 0, "Y": 0 },
            "End": { "X": 200, "Y": 500 }
        }
      ]
    }
  ]
}

Problems/Debugging

Logging

If issues are encountered, to get more verbose information in the logs, change the logging to the following:

"Logging": {
  "LogLevel": {
    "Default": "Information",
    "Microsoft": "Warning",
    "Microsoft.Hosting.Lifetime": "Information"
  }
}

This will output the full information log and help identify where things are going wrong, as well as displaying the confidence percentages from Deepstack.

Trouble Shooting

Not receiving notifications when one should have been detected

SynoAI is fast, but reliant on SSS to both notify it and return the current image. If you believe that a notification should have been triggered, then it's likely a configuration or performance issue with your configuration.

Image files are saved with a date/time different to the current local

Ensure your TZ environmental variable is set in your docker configuration.

Snaphots taken in quick succession always return the same image

Security cameras video stream includes I, P and B frames (See: Wikipedia). An I-Frame is like a jpg image, holding a complete snapshot of the scene. Between each I-Frame, the camera sends a bunch of P and B frames, each one holding only those parts of the scene that had any change / movement along time. I.e: In still scenes this saves lots of bandwidth.

At play back time, the video algorithm reconstructs the scene over time by first placing the I-frame as background and then composing each successive P or B frames on top, as time advances.

When SynoAI requests a snapshot from your NAS, Synology API just fetches the latest I-Frame. While this action is fast and simple, depending on your camera brand and configuration, the most recent I-Frame may be several seconds old and may not even include the actual moving object!

Some cameras are quite savvy on their bandwidth by really stretching the time between each I-Frame sent. I.e. DAHUA cameras brand got a configuration setting, labeled "SMART CODEC" which does just that when "ON". If this is your case, you should turn this "OFF", otherwise SynoAI may be fed old snapshots!

Snapshots are slow to fetch from SSS

Generally snapshots should be returned within a second or, at most, two. If taking a snapshot is taking too long, please try the following before reporting an issue.

Example slow log excerpt:

info: SynoAI.Controllers.CameraController[0] Front Courtyard: Snapshot received in 4167ms.

Camera names with spaces in cause the error "The camera with the name 'My Camera' was not found."

Spaces in URLs should be encoded using "%20". Most programs and applications, including SynoAI, handle this for you, but unfortunately the action rule on Surveillance Station does not. Therefor if your camera names contains a space, then you will need to ensure the URL in your action rule has all spaces replaced with %20.

Wrong: http://10.0.0.10:8080/Camera/Back Door

Right: http://10.0.0.10:8080/Camera/Back%20Door

Logs show timeout exceptions and SynoAI fails to initialise or the logs show crashes when passing snapshot requests to the AI

Example error:

System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.

The issue is a networking/firewall issue and is not a fault with SynoAI. This has been reported when using the dockerbridge network when all the containers are running on Synology DSM. In order to connect to all the containers, use the dockerbridge gateway IP address instead of the local IP from the Synology NAS.

"Failed due to Synology API error code X"

Common Synology Error Codes

DeepStack

FAQ

How do you set up a notification for each camera?

You need to specify a list of cameras against each notification. If a notification has cameras specified, then it will only send it to those cameras. If the Cameras list is empty or not specified, then the notification will be used for all cameras:

"Notifiers": [
  {
    "Type": "Webhook",
    "URL": "https://server/url_1"
    "Cameras": [ "Camera1" ]
  },
  {
    "Type": "Webhook",
    "URL": "https://server/url_2"
    "Cameras": [ "Camera2" ]
  },
  {
    "Type": "Webhook",
    "URL": "https://server/url_3and4"
    "Cameras": [ "Camera3", "Camera4" ]
  }
]