Person Tracker? - Githubissues

vikramgorla commented 5 years ago

I am looking at extending netatmo to have presence detection, stuck in some thought process

(I see #71 is somewhat linked to the same topic, but bit different)

I see all presence detection components are under device tracker as the principle means for presence detection is based on the location of the device. With the approach I am looking at, the presence detection is based on real presence of a human visible to the camera (e.g,. a kid may not necessarily have a device to associate with but can be defined as a person in Netatmo and it triggers an event when the kid is visible).

I am wondering from architecture perspective, what is the right component to use to implement person detection considering below

Person detection can be granular to a room which enables targeted automation based on where the person is in a house (May be applicable for device tracker with a certain amount of precision. e.g,. I have Unifi AP in every room in my house and Device Tracker could apply the same concept of room to devices looking at which AP is the device connected to)
I would like to have 1:1 mapping between person and device tracker (Owntracks tracks me when I am not at home and Camera detects the room where I am when I am in the house, Unifi controller to consider home when my device is connected to home WiFi,.) This can be done already between Owntracks and Unifi my adding mac to device created by Owntracks, but not when you have 3rd source, key question would be what could be the uid for person (each manufacturer can have it's own person identifier, just name+lastname may not be enough to be unique,.
We need enter/leave events for a person (e.g,. play a good morning message using TTS on my Google Home/Chromecast in that particular room when a person is seen for the first time in the morning,..)
Future compatibility if we define zone for each room..
Potential integration with face detection which is now based on camera

With Netatmo, you can link the seen person to a home, camera and a room already + retrieve the seen picture which is already face detected.

I see multiple options, need your view on what will be strategically right path

Extend Device tracker with a custom attribute for person_id and link it to person_id in camera, the logical issue is that not all entities are device.
Use Device tracker and have person_id from Netatmo as mac of device, downside is that you cannot have Owntracks, Controller based detection and person working at the same time.
Extend "someone_known" binary sensor and create a binary sensor for each known person to track them (downside is that you cannot link them to a device or define a picture by default without a template)
Create a new sensor called "persons at home" and have persons linked to it, same downside.

What do you think about having a component for persons deriving the same logic as devices? not sure if there were such discussions before?

balloob commented 5 years ago

I actually came to the architecture issues to talk about your last sentence: a component for persons.

For rooms, we're planning to add this soon for devices. Want to call it area to be a bit more generic.

The person component will represent a person that your instance is interested in.

A person can be linked to a user account
A person can be linked to 0 or more device tracker entities
A user has a location based on device tracker entities and their history. Location consists of:
- is_home: bool
- zone?: entity_id
- gps?: coords
- area?: area (only if is_home == true)
A user can be moving. If moving, we have direction, speed and mode of transportation.
A user can have a profile picture

The device tracker component should no longer be used to merge devices, this has never worked properly.

vikramgorla commented 5 years ago

Perfect, like this approach. Happy to help if you need some hands.

balloob commented 5 years ago

I think for MVP, we would get a person component for which we can configure per person:

first name
last name (optional)
user_id (optional)
list of device trackers (optional)

We then create a new entity per person that represents the latest device tracker state.

helto4real commented 5 years ago

I been working alot to make my trackers work in a appdaemon app. Now it is rock solid. Just grouping different devices wont do it. Using the last reported would be better but not enough I think. I find that different technologies are more suitable to detect home status and some if not home where you are (like zoning). The reason for this is that the gps trackers are not suitable to detect a 100% (or close) home status cause it sometimes is not fast enough or reliable enough. On android devices owntracks and such tend to suffer from battery savings and report data not fast enough to use as single tracker. wifi/BT are normally very fast to detect home status.

Let me give an example how I would like the person trackers to work.

wifi/Bluetooth if one of them is "home" then person is home
wifi/BT not home, check gps type tracker and use value for it if it been reporting last 5 minutes (configurable timeout)

Ok how do we make this general? Maybe we could set home prio flag in config for a specific device. Rest is we need to see gps capability device automatically or just have home prio for every device that are not of gps type?

This is my two cents on the issue. I know we should always make it simple but not too simple that will make the tracker not usable for automations depedning on home status/occupancy.

Thanks

balloob commented 5 years ago

We already have source type for device tracker entities. So I agree that using a router/bt/wifi find take priority over GPS.

DubhAd commented 5 years ago

My current setup is that I'm separately using GPS + WiFi + BT and working on (mostly) a majority wins logic. I'm getting no false results, since if (say) the WiFi is asleep the odds are that BT still shows at home and the last GPS update also did. This also allows for when somebody is at the next door neighbours, where the GPS may show home, but nothing else will. The downside is that arriving home can be fractionally delayed with just that logic, so it requires that arriving home can be triggered by just WiFi or BT.

So, a person tracker that allows us to link a person to trackers, and define our priorities and what determines home/away/arrived, would be great.

raccettura commented 5 years ago

I'd like to suggest incorporating some ideas from: https://philhawthorne.com/making-home-assistants-presence-detection-not-so-binary/

In particular:

More than boolean state. It can be handy to know the difference between "just left" and "extended away" and making those deeply integrated and standard would be a huge improvement. For example you may not want to adjust your thermostat for "just left", you like would for "extended away".
Bayesian learning could be handy, though perhaps it could be smarter if HA could actually prompt for user confirmation early on in a "learning mode" to help adjust the values. The hardest part of Bayesian is adjusting the values. For example Bluetooth will notice I'm home before WiFi device detection will, but Bluetooth is also sensitive and will occasionally see me leave when I haven't really left. Getting that right mix is hard.

jjlawren commented 5 years ago

We already have source type for device tracker entities. So I agree that using a router/bt/wifi find take priority over GPS.

This unfortunately isn't a safe assumption in practice. For example, the Unifi controller queried by its device_tracker component has a relatively long timeout that keeps entities as home after they have left. It's very accurate when a device arrives, but other components would be more accurate to determine when leaving.

yottatsa commented 5 years ago

@raccettura It seems that proximity is what you want.

Hedda commented 5 years ago

Out of scope for a minimum viable product of this feature but FYI; one tip for related GUI interface part of a such "Person Tracker" concept would be to in the long-term consider looking at borrowing ideas from the user interface of the Fing app when you have the accompanying Fingbox. This as the "Fing" app when you have a Fingbox on your network has a very nice GUI interface and user experience for adding multiple persons to your home and then associating devices to those persons.

Fing calls this feature "Digital Presence" and you have to give it to the Fing.io team that they got excellent user experience engineering skills, and here is an overview page how the GUI layout of this function is in the Fing app when you have a Fingbox. This describes the entire process of adding a new user and assigning devices:

https://help.fing.io/knowledge-base/digital-presence-detection/

Here is a video which goes through the app GUI and features

https://www.youtube.com/watch?v=wsvwSNunqbg

Also check out their description as well as their video describing of the related " Digital Fence" (geofence) feature:

https://help.fing.io/knowledge-base/digitalfence-feature/

https://www.youtube.com/watch?v=IYIW8KVJ-iY

While the Fing app with Fingbox primary use this specific core feature of associating devices to those people for Parental Control functions in a household, its user interface to achieve this part of an adding people and then associating devices is very user-friendly from and end-users point-of-view.

https://help.fing.com/wp-content/uploads/2018/12/Digital-presence-e1544633161743.png

Summary:

Create Users & Assign Personal Devices.

One of the most important Fingbox features is Digital Presence. To enable Digital Presence, you need to create users and assign personal devices to each of them.

From the People tab click on the circle with a plus (+) sign to create a new user. Choose the devices belonging to that person; for example, their mobile phones, computers and tablets. Then make sure to assign a device that represents that person’s presence – this is the device that reflects their movement in and out of the office. A smart phone, for example, is likely to always be on them, whereas their desktop computer stays in the office at all times.

To select the PRESENCE Device, click on the user in the user carousel and then select Devices in the pop-up. Find the device you wish to turn into the PRESENCE device and click the CHANGE button on the right-hand side of the device. This will then give you the option to state that it always travels with that user. You will then see the CHANGE become PRESENCE.

Some additional pictures from the Fing app

Landrash commented 5 years ago

Goes against what's suggested but user_id usually is the only thing that doesn't change in most systems. Since people can change their name. Wouldn't a user_id be simpler since names can often contain characters that needs to be normalized but it might be desired to have these still shown in the UI correctly?

tboyce021 commented 5 years ago

Random thought: Is this meant to be strictly used for people or could it also be used for pets? In that case, "Person" might be a bit misleading, in the same vein that "Area" was chosen over "Room" to be more generic.

balloob commented 5 years ago

The person component has been merged as part of 0.87

MartinHjelmare commented 5 years ago

0.88

home-assistant / architecture

Person Tracker? #72

I am looking at extending netatmo to have presence detection, stuck in some thought process