Feature request - Person detection

[FeikoJoosten]

Hi, been loving this addon so far. Thanks for all of your work! I was wondering if it's possible to add an additional attribute to the last motion event sensor which indicates if the motion event has detected a person. After taking a quick glance in the API this is based on, this feature should be possible.

Obviously if this is out of scope, then that's fine as well.

[tsightler]

I would have to look at it in more detail as I don't remember the person detection being part of the ding event. I believe the option you reference is only for querying camera events, which means I'd have to take the active ding then try to search for the matchin event to determine if the motion ding was person or not. Not saying no, just need more research, maybe it's there and I just didn't notice it.

Unfortunately this is queued up behind completing snapshot support and then work on the promised alarm arming w/sensor bypass option, so it might be a while before I can look at it as my time is very limited right now.

[FeikoJoosten]

No problem at all, take your time!

I took another look as well and you seem to be correct that it's not included in the active ding event. Which'll mean you'll have to have to query the events. Doesn't sound like an ideal solution either.

[tsightler]

I've been researching this a little bit and I'm struggling to see how to make this work. Yes there are event properties which appear they would be used for this, but so far I've not been able to find a single event on any of my cameras where the person_detected property is set to anything other than null, even though, for example, I have my doorbell set to only alert on people.

As far as I can tell, person detection with Ring cameras at this point is only about reducing notifications, but the notifications and events you get are the same regardless. Can you tell me if there's some way to identify motion events from person events in the Ring app anywhere? If not, I think I'm going to have to come to the conclusion that it's just not possible. In my own testing I can find nothing in the Ring app that shows a person detected event different than a motion detected event, only that you can set "People only mode" which basically just filters events/notifications from being received.

Even if the event did have the property set correctly, I'm not sure how useful it would be since the event entry itself lags behind the ding quite a bit so it would take time to determine if the event was a person event or generic motion event.

[FeikoJoosten]

Thanks for looking into this!

Can you tell me if there's some way to identify motion events from person events in the Ring app anywhere? The only thing I can see is that if you open the event viewer, you can see if a motion detected a person or not.

Apart from that I agree that the person detection is just a notification filter.

I'm not sure how useful it would be since the event entry itself lags behind the ding quite a bit so it would take time to determine if the event was a person event or generic motion event. This is something I wasn't keeping in mind and is unfortunately quite a shame.

Feel free to close this if you don't think it's possible at all. Once again, thank you for looking into this.

[tsightler]

Interesting and thanks for sharing, I don't see this at all with my cameras. I wonder if my cameras are just too old or something? I have two Floodlight cames, to Stickup Cam Elites, and a Doorbell Pro so they are all wired power devices and I would think they should be able to support person detection.

Could you share a little more detail about how you configured this for your camera in the Ring app? The only option I see on my cameras are the normal sensitivity slider, and the option to enable "People only mode". Is that what you have configured? I have this set on a couple of my cameras but no matter what I only get "Motion" events.

[FeikoJoosten]

I have a ring doorbell 2nd gen. Currently using it in battery mode. And all I’ve done is enable the person detection mode. This seemed to have changed the way ring sends me their notifications. Either with the person detection, or with the motion only.

[tsightler]

Just to be 100% sure, it's the option below, right:?

https://support.ring.com/hc/en-us/articles/360032947972-Refining-Your-Alert-Notifications-with-People-Only-Mode

In a few days if I put together a special dev branch that collected some extra debug data on dings, would you be willing to run it and share that data with me via email? There is a field on the ding that I believe may contain the information on person detection, but since I can't get any of my cameras to actually send person detection events, I can't prove it.

[FeikoJoosten]

Yep it's indeed the feature below. I did notice this at the bottom of that page

Does everyone have access to People Only Mode?

No. People Only Mode is only accessible to those neighbors who have signed up for a Ring Protect Plus or Ring Protect Basic > Plan. People Only Mode is also available during the 30-day trial period that begins when a device is first activated. After the > trial ends, a Ring Protect Plan is required to use the feature.

I myself do have a Ring plan, do you have this as well? Otherwise it might be the country you're from, maybe it has some privacy laws? (Just guessing at this point).

In a few days if I put together a special dev branch that collected some extra debug data on dings, would you be willing to run it and share that data with me via email?

Yes of course, would be no problem at all!

[tsightler]

Has, I have the full Ring Protect plan. I'm in the US so I can't imagine anything is stopping Ring from a legal perspective since we have basically no privacy rules, and my state certainly doesn't have any.

Anyway, I'll see if I can put together a quite piece of test code for you to try. Basically, I just need to get a "ding" event when a person is detected and see if it's in any way different than a regular motion ding. I see one field that is null on my cameras that appears it might show this data. Thanks for your help!

[FeikoJoosten]

I don't see why it wouldn't work for you either then haha. Maybe contact Ring? (You can tell them that someone in the Netherlands receives this information in their app).

Basically, I just need to get a "ding" event when a person is detected and see if it's in any way different than a regular motion ding.

Yeah, figured it'd be something like this hehe.

Thanks for your help!

I should thank you hehe. You're the one putting in the effort 😉

[tsightler]

An hour on the phone with Ring support and I'm not sure I ever convinced them that they even have person detection feature. I had to send screenshots showing the feature and showing it working on other accounts vs mine with the same devices. After an hour basically we had accomplished toggling the feature on and off to see if that would fix it and wanting me to reinstall the app, even though the problem is obviously not app related. Ring support is a JOKE!!

[FeikoJoosten]

Wow that sounds like an amazing time...

[tsightler]

So, as Ring still insist that person detection isn't a thing for me, I've given up on that path and finally just added a little bit of debug code to the dev branch to output the full ding event and see if there's anything in there that identifies when a person is detected.

If you are using the addon or docker install you can test the dev code by leveraging the branch feature. In your config simply change the "branch" config option from "addon" to "dev" and restart the addon. Startup will take a little longer as it pulls the latest dev branch from github and builds dependencies. Then if you can create some motion events, which should now log the fill ding data. Once done, you can go back to the standard build but simply changing back to "addon" branch in the config.

If you don't want to post the logs here you can send them to me via gmail at the same username.

[FeikoJoosten]

Okay, so I'm currently running your development branch, but I'm guessing I need to change a config setting somewhere because I don't see any logs showing at all.

This is my config in HA for the addon:

enable_cameras: false
snapshot_mode: disabled
enable_modes: false
enable_panic: false
enable_volume: false
mqtt_host: <auto_detect>
mqtt_port: <auto_detect>
mqtt_user: <auto_detect>
mqtt_password: <auto_detect>
branch: dev
ring_token: ''
location_ids:
  - ''

And this is my full log: (removed my town and location id from the log)

-------------------------------------------------------
| Ring Device Integration via MQTT                    |
| Addon for Home Assistant                            |
|                                                     |
| Report issues at:                                   |
| https://github.com/tsightler/ring-mqtt-hassio-addon |
-------------------------------------------------------
Updating ring-mqtt to the development version...
Cloning into 'ring-mqtt-dev'...
Installing node module dependencies, please wait...
-------------------------------------------------------
ring-mqtt.js version 4.4.0
Node version v12.21.0
NPM version 6.14.11
git version 2.26.2
-------------------------------------------------------
Using discovered MQTT Host: core-mosquitto
Using discovered MQTT Port: 1883
Using discovered MQTT User: addons
Using discovered MQTT password: <hidden>
-------------------------------------------------------
Running ring-mqtt...
2021-03-13T19:20:12.177Z ring-mqtt Using configuration file: /data/options.json
2021-03-13T19:20:12.180Z ring-mqtt Reading latest data from state file: /data/ring-state.json
2021-03-13T19:20:12.247Z ring-mqtt Attempting connection to Ring API using saved refresh token.
2021-03-13T19:20:14.298Z ring-mqtt Connection to Ring API successful
2021-03-13T19:20:14.299Z ring-mqtt Starting connection to MQTT broker...
2021-03-13T19:20:14.352Z ring-mqtt File /data/ring-state.json saved with updated refresh token.
2021-03-13T19:20:14.475Z ring-mqtt MQTT connection established, processing locations...
2021-03-13T19:20:15.479Z ring-mqtt --------------------------------------------------------------------------------
2021-03-13T19:20:15.479Z ring-mqtt Found new location CITY with id LOCATION_ID
2021-03-13T19:20:15.480Z ring-mqtt --------------------------------------------------------------------------------
2021-03-13T19:20:15.480Z ring-mqtt Ring location/device data updated, sleeping for 5 seconds.
2021-03-13T19:20:20.482Z ring-mqtt No devices found for location ID LOCATION_ID

I've also checked the supervisor logs, but couldn't see anything ring related there either.

[tsightler]

At the current time the development version has only a single line difference from the production version to output the ding data to the debug log, so I'm not sure how it could possibly work differently from any standard install of the addon. OK, technically two lines, but one just moved a little.

Since we are talking about cameras you'd definitely need to at least set enable_cameras to true.

[FeikoJoosten]

Ah yeah using the camera might be useful haha. Anyway, do you know of a way to retrieve the full log? The log in the addon view is cut-off.

[FeikoJoosten]

Alright, using portainer I was able to retrieve the logs. This pastebin expires after 1 week. https://pastebin.com/yucM3zjY

Here's also a screenshot of the ring history view to give you the timestamps. (note, for some reason portainer uses a different timezone. So subtract the timestamps of the screenshot with one hour to get the portainer timestamp. (So 21:17))

[tsightler]

Perfect, that's exactly what I needed, specifically this one single line from the ding even itself:

detection_type: 'human',

So basically, if this is null value, it's normal, while if a person detected, the value is 'human'. That seems pretty easy to work with and I can detect a person event in real time.

Now the question becomes, what is the best way to surface this data. Should it just be an attribute on the existing motion sensor entity, or should it be an additional sensor. One challenge with using an new sensor is trying to figure out what class of sensor it should be as it doesn't seem correct to have a second sensor of class motion. Maybe "presence" but I think that's more about being home/away vs person detected.

My gut says I should simply add an attribute like "person_detected" and set it to true/false based on the last event received, but I'm open to feedback on this.

[FeikoJoosten]

Yep I noticed that specific line myself as well haha. Still odd that you can't reproduce that on your end. Possibly it's Ring trying out different features per region or something.

But to come back to the topic, I agree that an attribute would be best in this case. I don't think an additional sensor makes much sense especially since not everyone seems capable of receiving the attribute at all.

[iceman23e]

@tsightler Another idea would be to create sensors like the Ring integration. Last activity, motion and ding with the attributes about that activity, which you could include something like Human true/false.

[tsightler]

Maybe, but I'm not sure that makes the most sense for this project which is not HA focused.

[iceman23e]

Apologies, I thought since you were talking entity attributes and sensors and Feiko is using the HA addon, it might be worth a mention.

In any event, thanks for all your work on this project.

[tsightler]

Sorry, that was an incomplete comment that was posted accidentally. There's nothing wrong with your idea at all, I really just meant that I have to think about it not just from the HA perspective but almost what makes sense for the generic MQTT use case. While this project used used quite heavily with HA, and I always aimed to make it interoperate with HA as easily as possible, the core project is about interfacing with Ring devices via MQTT and there are quite a lot of people that use it generically for other home automation platforms as well.

[FeikoJoosten]

Personally I'd say just make it an attribute. It'll be supported on all platforms, and it allows for easy integration for all users.

[tsightler]

Pushed 4.5.0 which includes a PersonDetected attribute on the camera motions sensors. Since Ring still hasn't managed to even acknowledge that such a feature exist for my account I was unable to test this, so I tried to just do it based on logs. I'm definitely a little concerned that it might be an issue if Ring send mulitple ding events (a non-human and a human detect) ding.

If someone that actually has this feature could test it and let me know if it works or needs any tweaks I would appreciate it.

[iceman23e]

Tested on my battery doorbell and I only received one ding that a person was detected and the attribute showed true. Seems to be working properly. Thanks for all your effort.

[tsightler]

Thanks for testing. Based on this initial feedback I'll close this for now, someone can open a new issue if tweaks in behavior are needed. Note that PersonDetected should always show the status of the most recent motion ding received.

[EvanKnowles]

Sorry to bring up an old thread, but I'm not seeing any PersonDetected information on a Spotlight Cam. Is it only supported for the Doorbell? Or is there a specific topic I need to subscribe to? I'm currently subscribing to motion, info, ding and event_stream.

[EvanKnowles]

Nevermind, found it by inspecting logs - for anyone else looking, I needed the motion/attributes topic on the camera.