Improvement Proposal: Integrating Features from FIND3

[jleinenbach]

Improvement Proposal for Bermuda-Code

Is your feature request related to a problem? Please describe. I recently came across FIND3 by Schollz, an advanced project for indoor positioning that seems to have been discontinued. It had an Android app that was unfortunately removed from the Play Store. The project supports not only WiFi signals but also Bluetooth and magnetic fields for determining precise indoor locations. It would be great if we didn't have to reinvent the wheel by starting from scratch.

Describe the solution you'd like I suggest we integrate some of FIND3's advanced features into Bermuda-Code. FIND3 offers:

• Multi-source data support including WiFi, Bluetooth, and magnetic fields.
• Machine learning capabilities with support for ten different classifiers.
• APIs for sending fingerprints (/track and /learn) and MQTT endpoints, which are backward compatible.
• A permissive MIT license that allows us to freely integrate its components.
• Comprehensive documentation and community support available via GitHub and Slack.

Describe alternatives you've considered We could develop similar features from scratch, but this would require significant time and resources. Alternatively, we could look for other existing solutions, but FIND3 already aligns well with our needs.

Additional context FIND3's modular architecture includes a data storage server, a machine learning server, a command-line tool for fingerprint collection, an Android app, and ESP code for fingerprint gathering. Integrating these components could significantly enhance Bermuda-Code's capabilities. Check out FIND3 at https://github.com/schollz/find3 and its documentation at http://www.internalpositioning.com for more details. Notably, I haven't found any existing integration of FIND3 with Home Assistant, which presents a unique opportunity for us to fill this gap.

[agittins]

I can't see anything particularly useful there.

Are you aware of anything specific that find3 does that's unique or different in some way? The example clients just measure rssi of APs.

FIND3 already aligns well with our needs.

I don't see any evidence for this bold claim, and the website doesn't give me much to go on as far as working out how well it worked. Also it's mostly written in Go, not python, so any code re-use would involve porting, so the squeeze would have to be worth the juice.

There are plenty of indoor localisation projects around, it looks similar to most, as does Bermuda. Bermuda's key points are that (a) HA's bluetooth_proxy framework is an excellent basis to build this sort of thing on top of, and (b) I feel that the prior art left some ideas unexplored or under-optimised.

I'm having trouble identifying anything actionable in this.

[jleinenbach]

I found it interesting that it also includes WiFi APs, and they claim it uses machine learning (but how? The user will have to train it, I guess?). I noticed that with Bermuda, my BT proxies one floor higher (with concrete in between) sometimes appear to be closer than the BT proxies 2 meters away in the same room. I think this is because temporarily stronger signals are interpreted as "getting closer," and therefore, no average values are formed. However, since WiFi signals are blocked differently by walls and ceilings, including them as a second opinion might be useful.

Machine learning might be an overstatement, but I like the idea of training: I could imagine calibrating a room. During calibration, the proxies, regardless of their location, focus only on a single device. This way, it becomes clear which RSSI values are falsely high (due to thin walls, etc.) or falsely low (due to furniture in the way). Once a device is detected in a room, this calibration profile could be applied to the proxies, preventing the device from being falsely identified as being in an adjacent room too quickly.

[agittins]

they claim it uses machine learning (but how?

Hehe yes it's a bit like when people say "we use the ~ blockchain ~ (sparkles!)" and you just know that it's not actually a critical part of the process :-)

I agree about more data being (potentially) more useful, but only a small subset of devices will have wifi ranges available (eg, there's no inter-device rssi, only device-to-ap, and most beacons, many watches etc don't have wifi at all). So it might be possible to augment our data with wifi stats at some point, but it would only be useful after we've got as much as we can out of the bluetooth data - and we still have a lot of improvement available on that yet.

The trilateration support should help a lot with that, as it will be considering each measurement in context with the other available measurements - this is at the root of some of the core design goals of Bermuda, to make use of the aggregate of measurements (sort of a more-garbage-in-equals-better-garbage-out delusion).

Minor point - there are in my opinion very few if any "falsely-high" rssi values, only "falsely-low" :-) This is why traditional filtering like kalman filters aren't (in my opinion) well-suited to tracking rssi distance, since they assume the noise is symmetrical, which it most definitely is not.

I'm sure there are applications for machine learning in this, but my opinion is that application of thought-out reasoning will be far more successful to start with, and I suspect that the results might be good enough that further refinement through ML wouldn't be necessary.

Bottom line is that right now Bermuda's detection is a minimum-viable-product of saying "what's the closest proxy? OK, we're in that room then" which is the very least one could possibly do, yet for some reason it seems to be out-performing other solutions in many cases - so I'm going to stick to my design goals for now, given that the ML attempts that I've seen have all been pretty ordinary.

[jleinenbach]

Yeah, the thing with the nearest proxy is especially problematic when the proxy in the room is relatively far away, and the one on the floor above is closer. It seems not even the concrete ceiling can help. I specifically solved this for the living room by installing two proxies.

Attenuation seems to be a bit of a double-edged sword: if I set it lower, the distances are estimated to be larger more correctly. If I set it higher, the incorrect proxies are filtered out better, even though they appear closer. Still, it bothers me that the kitchen, which is almost 10 meters away from the living room, is shown as just 1.5 meters for the iPhone (IRK connection). The second proxy in the living room solves this issue, but with just the first proxy, the iPhone often switches between the living room and the kitchen. And that's despite the kitchen being around the corner from the living room in a stone house.