[FEAT] Ring-MQTT integration

[scholy]

I wonder if it's possible to integrate Ring-MQTT with double-take to process either the live rtsp:// when a motion is detected or the video file stored in AWS following a motion event (about a 1-2min lag)? I know Ring-MQTT doens't play well with frigate, but is the MQTT event data provided by the integration sufficient for double-take to trigger deepstack analysis and is the media (rtsp or mp4 file) suitable for deepstack to analyse?.

Ideally, I'd like to process facial recognition for people at my front door and send notifications via HA.

I have armcrest and dahua cameras working wonderfully, love the HA addon!

[jakowenko]

Hey @scholy, do you have an example of what the ring MQTT payload looks like? Seems like this should be possible.

Thank you for the compliments, I'm glad it's working well for you.

[luzik]

In my humble opinion this should be done outside double-take. Double-take can just provide service call for triggering such action. There are many Ring-MQTT out there, it will be hard to support all of them.

[scholy]

@jakowenko, there's a number of messages that come through.

Regular snapshot, configured in the Ring app (off, 30sec, 60sec, 180sec):

Message 0 received on ring/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/camera/54e01980c39e/snapshot/attributes at 6:32 PM:
{
    "timestamp": 1647415967
}
QoS: 0 - Retain: false

Status message, posted every 5min

Message 25 received on ring/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/camera/54e01980c39e/info/state at 6:40 PM:
{
    "firmwareStatus": "Up to Date",
    "lastUpdate": "2022-03-16T07:36:23Z",
    "wirelessNetwork": "mySSIDname",
    "wirelessSignal": -53,
    "stream_Source": "rtsp://_ringusername:password_@03cabcc9-ring-mqtt:8554/54e01980c39e_live",
    "still_Image_URL": "https://homeassistant:8123{{ states.camera.front_door_snapshot.attributes.entity_picture }}"
}
QoS: 0 - Retain: false

Motion state, posted on state change to ON:

Message 48 received on ring/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/camera/54e01980c39e/motion/state at 6:47 PM:
ON
QoS: 0 - Retain: false

Motion attributes

Message 49 received on ring/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/camera/54e01980c39e/motion/attributes at 6:47 PM:
{
    "lastMotion": 1647416868,
    "lastMotionTime": "2022-03-16T07:47:48Z",
    "personDetected": true,
    "motionDetectionEnabled": true
}
QoS: 0 - Retain: false

And resulting video file from motion, delayed from motion event based on length of recording and upload time to AWS

Message 51 received on ring/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/camera/54e01980c39e/event_select/attributes at 6:49 PM:
{
    "recordingUrl": "https://filestore-734410259931-us-east-1-prod-data.s3.amazonaws.com/7b0f2411-308b-407f-a56d-11b2c66da475.mp4?X-Amz-Security-Token=_aTokenWasHere_",
    "eventId": "7075601558151086422"
}
QoS: 0 - Retain: false

[jakowenko]

Are these from the official Ring app? If so, I don't mind supporting Ring, but if it's using something third party along the way, it might not make as much sense.

Based off this, it seems like you'd want to listen to the motion/state topic for the ON state and then process the images. Is there a consistent JPG URL that is available that could then be polled at some sort of interval when motion is detected?

[divemasterjm]

i'm also wondering if is possible, or just how to send from mqtt or homeassistant an image to be recognized, cause i have 2 cameras out of frigate and want to have them with doubletake

[MooreSi]

I have exactly the same query, has anyone managed to make this work?

[leccelecce]

I've done some early testing with ring-mqtt using snapshot images rather than videos. It works...about as OK as I would expect.

I think the key thing to read here is the ring-mqtt documentation section which explains the limitations of the cameras and how you should not expect them to work for continuous streaming: https://github.com/tsightler/ring-mqtt/wiki#camera-configuration

However, given that I can see snapshots updated periodically in the Ring app / Home Assistant, I thought I would at least try and see how well I could tie it in to Double Take.

My setup: Various Ring cameras (Doorbell 3 Battery, Wired Indoor Cams) -> ring-mqtt -> mosquitto -> Home Assistant

What I wanted to do: somehow pickup the ring-mqtt snapshot image, send it to Double-Take, which could then publish the detection back on to mosquitto for consumption by Home Assistant via mqtt

I spent a bit of time setting up a custom flow in Node Red to try and link ring-mqtt and Double-Take, only to discover I could get it working out of the box without Node Red thanks to some of the excellent work already done by jakowenko :-)

It really is as simple as:

1. Find the ring-mqtt topic name for your camera, taking the snapshot/image topic (which is the binary data of the jpg)

ring/xxxxxxxxx-xxxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx/camera/yyyyyyyyyyyy/snapshot/image

2. Configure a manual camera in Double-Take:

cameras:
  lounge:
    snapshot:
      topic: ring/xxxxxxxxx-xxxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx/camera/yyyyyyyyyyyy/snapshot/image

3. Double-Take will then monitor the ring-mqtt topic and when a snapshot JPG is posted, automatically run recognition on it.

Observations:

• Ring cameras (at least the ones I have) are 1920x1080, but the snapshots only appear to be 640x360. At a reasonable distance this means the face box can be very small (sub 100x100) which reduces accuracy. I assume a higher res camera would have a larger bounding box - though some other research suggests that people tend to use lower resolution for detection so I'm not sure how accurate that is.
• ring-mqtt does explain how the snapshotting works, and does make attempts to request a snapshot when a motion event begins. However whether it gets one in good time, and from a good angle, is another matter. Ultimately how useful it is is going to depend on working out something reliable here.

[update] After some further testing, one issue I have is that Ring doesn't seem to apply much intelligence when selecting a snapshot photo. This is a problem because if it starts detecting someone a reasonable distance away, or while they're not angled towards the camera, the snapshot can be pretty unhelpful for detection processing. This is always going to be a struggle vs something like Frigate monitoring a stream continuously and checking multiple frames.

[leccelecce]

For awareness, some separate discussion on this via the ring-mqtt maintainer is going on here: https://github.com/tsightler/ring-mqtt/discussions/501