FIND3 Passive Integration

[ptdgh]

Is your feature request related to a problem? Please describe. First world problem in the current requirement of a scanning client in every room that I require room level automation.

Describe the solution you'd like I would like to have my cake and eat it too. Room-assistant's simplicity and speed is both its strength and weakness; however, I feel that by leveraging FIND3's extensive classification and learning system the end user should be able to extend room-assistant's functionality/accuracy and reduce the number of clients required to cover the required rooms and/or area.

Describe alternatives you've considered Direct alternative is FIND3's native find3-cli-scanner in passive mode. This solution while functional is a bit slower and for whatever reason I seemed to have stability issues that I do not have with room-assistant. FIND3's native client also lacked BLE functionality which I have started toying around with and was happy that room-assistant supported natively.

Additional context I am currently testing my own room-assistant/FIND3 integration via node-red watching mqtt topic changes and grabbing data via the room-assistant's api to reformat and post to FIND3. This appears to be functional at a glance but my FIND3 instance needs more samples, learning in various locations, and perhaps whitelisting some stationary BT devices in the rooms to have a better fingerprint. I suspect that due to bluetooth range limitations additional clients may still be required, but if it's not one in every room I will call it a step in the right direction for my case.

Having tested with wifi fingerprints in the past, both in active and passive - I am ready to accept that a few more clients and bluetooth's range limitations is actually a plus until something better comes along (wifi rtt?). In the meanwhile this appears to be a interesting solution/integration that I thought would be of interest to others.

[mKeRix]

FIND3 has always interested me, it's a great tool - but it never supported my use cases, which is why I wrote room-assistant in the first place. I wasn't even aware of the possibility to integrate up until now. Certainly very interesting and would likely be a better solution for reducing the amount of required scanners than just doing triangulation. Maybe it could even help with overall accuracy.

Thanks for submitting the idea, I'll note it down for further research on a rainy weekend. :)

[mKeRix]

After looking at this a bit more in detail I want to try and replicate the ML part of FIND3 in room-assistant. While writing the integration from room-assistant to FIND3 shouldn't be particularly difficult, the FIND3 project does not seem actively maintained anymore. The last release was done in 2018 and there have been no real code changes or accepted PRs lately. Replicating this within room-assistant would give me the power to adjust the details of the detection and also make the feature easier to use.

If you can think of a reason why I shouldn't try this (or why it is way more complicated than just writing up something with TensorflowJS) then do please let me know.

[ptdgh]

I would be curious what you come up as I suspect your method may be a bit more consistent considering you have essentially weighted locations/anchors per scanner vs FIND3's fingerprint/classification system which seems to work, but is a bit frustrating and inconsistent at times considering interference, noise, and rssi variability.

[jaburges]

@mKeRix check this out: https://github.com/st0nedB/rooms

It uses a training process (and a mobile app). In this example the sensing is reversed so the phone does all the work (not ideal), which is why i prefer room-assistant

However you add a room, move about and the phone captures training data. Export the data, run it through a python AI model, and reimport the results to the phone.

What would be cool is if you could do something similar in room-assistant.

Set up the Nodes/Rpis and this time just label them as node numbers (not rooms). Then either from the App, or from maybe a separate AI docker container, create a bunch of rooms. Set one of the rooms as active, and then move about in that room with a known device (all the while the training docker is capturing the values of the nodes whilst you are in that room). After 120 samples a file is generated. Rinse and repeat for each room.

Now run the training model, and update the workflow. My thought is that you would have options here.

No AI (just RPi nodes and they are room based)

AI and the additional docker that listens to the RPis / Nodes and using the AI model infers where you are and just outputs a room location and maybe the status of all the nodes and AI docker for tracking

[davidrustingha]

I don't know if this is still an active project, but I would love to help out. I currently have 2 rpi 0's running RA and will move to a bigger house around march. I'm pretty sure I'm gonna buy more rpi's

[mKeRix]

This project is still active and this feature is still on my roadmap - I've just been putting my available time into another issue that is particularly tricky the last couple weeks. I know that @jaburges is also really interested in getting this to work, maybe he has already made some progress.

Other than that, I can share a few initial thoughts of how I imagined the implementation if you're interested in picking this up. :)

[davidrustingha]

I don't think I can be of much help with developing. I rely on the skills of others since I've never build custom components like these. But I have quite the smart home for giving these things a good test run.

EDIT: But I'm always willing to learn more so show me the goods ;-)

[mKeRix]

Testing is also very valuable - my test setup only spans a few rooms in an apartment, so I can't cover all conditions (like more rooms, multi-story, etc). :)

Generally I imagined the setup to be another integration that can be enabled at will. In the first step this integration should expose at least two endpoints the room-assistant API: one that enables training data collection and one that starts the model training process. For data collection I would suggest to bunch together the latest RSSI values that are not beyond the timeout each second or so for each tracked device and add the room you are training for as the "expected result". For the model the project linked above should be a good start and relatively easy to implement via tensorflowjs.

The relevant RSSI values can be pulled from the entity updates once I've updated that system a bit (which is next up on my plans). The integration can then just add its own new entities for all the room presence entities it's tracking, which then contain the predicated states (and maybe the certainty in the attributes).

[davidrustingha]

Sounds pretty complex to me though. I do think I understand the way it's supposed to work. Let me know when I can be of help. Around march I will upgrade from an apartment with 2 pi 0's to a house with 3 levels and at least 5 pi's scattered around

[mKeRix]

Since the discussion on this has drifted into a whole different topic than the original issue was for I created a follow-up issue for tracking of the machine learning implementation: #507.