Unifi - presence detection stops working if wired clients are listed in unifi_data

[GatoPharaoh]

Home Assistant release with the issue:

103.0

Last working Home Assistant release (if known): unknown

Operating environment (Hass.io/Docker/Windows/etc.):

HA in a venv on Raspbian Buster / RPi4B **Integration:**

https://www.home-assistant.io/integrations/unifi/

Description of problem: mac-addresses of devices once connected to an Unifi AP will be added to the file unifi_data in /home/homeassistant/.homeassistant/.storage as wireless_devices

Once a device is listed in unifi_data and goes back to be a wired_client (e.g. when it's connected to an AP from a different manufacturer), presence detection stops working. Devices listed in unifi_data will stay at not_home. Removing the mac address from unifi_data and restarting HA will fix this issue temporarily. However, if a device reconnects to an Unifi AP, its mac address will be readded to the list in unifi_data (and will therefore not be recongnized as homeagain)

See also https://community.home-assistant.io/t/unifi-presence-detecion-issue-with-unifi-data-and-wired-clients/155497

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

none as added via integrations

Traceback (if applicable):

Additional information:

[probot-home-assistant[bot]]

Hey there @kane610, mind taking a look at this issue as its been labeled with a integration (unifi) you are listed as a codeowner for? Thanks!

[Kane610]

There is a known bug with UniFi that wireless clients going out of range from the Access points gets marked as wired. The Only thing I could do here would be to introduce an option to skip the wired bug fix.

[GatoPharaoh]

if this is easy to implement, I'd take it. At the moment, I'm blocking Home Assistant from writing to unifi_data, which has reduced false presence detection significantly.

thanks for the input and your work!

[Mike1082]

This is very interesting. In older versions of the Unifi Controller, it would take about 15 minutes for a wireless device to "disappear" from the Clients view (but would remain in the list as a wireless client). For that 15 minutes, Home Assistant would be waiting for the update that the device was gone (at least that seemed to be my experience). With this new version (and I suppose this is part of the bug described above) the device now appears as "USG" under the AP/Port column:

Good find on the MAC addresses stored in that file, OP. This explains why my Unifi Device Tracker is not working.

[finity69x2]

I've jut today realized that I've been experiencing this bug as well since at least August when I switched to using Unifi gear along with my old ASUS routers as AP's.

@Kane610

The Only thing I could do here would be to introduce an option to skip the wired bug fix.

I assume the reason this is needed is to allow the option of tracking only wireless clients?

So as an alternative would it be possible to write the integration to always track the device as "wireless" after the device is added to the unifi_data the first time no matter how the Unifi controller reports the actual mode that the device is connected?

IOW, if the device is listed in the "wireless_devices:" list then that device will forever be treated as wireless no matter what the controller reports (and therefore will always still be tracked as wireless)?

I think that would be a reasonable compromise to be able to only track wireless clients and also allow the use of non-Unifi hardware. On the rare occasion that a previously wireless client would go to wired and the user no longer wants to track that client then it's trivial to just disable that one entity in the entity registry.

[Kane610]

@finity69x2 the problem is that with the wired bug AFAIK the timestamps gets messed up so they can't be trusted. I haven't actually experienced the bug myself, but according to others the current solution solved it for people with "pure UniFi" systems. What you suggest I already do to the extent it is possible, which is to mark a device as known to be wireless but at the same time if it shows as wired we can't trust UniFi reporting anymore.

The real solution would be for Ubiquiti to acknowledge this bug and fix it.

[finity69x2]

@Kane610

Would there be any benefit to you to post my debug logs?

I finally started to deepdive into figuring out why my android phone would never connect and was eventually directed here. but prior to that I had set up debug logging to try to see if that would lead me to any conclusions. I've got a ton of logs showing both before I manually edited my MAC from the list (when it wasn't working) and after (and it is working...for now...)

[finity69x2]

@Kane610

I guess I don't know what you mean when you say "if it shows as wired we can't trust UniFi reporting anymore." You mention something about "AFAIK the timestamps gets messed up" but then say that you've never experienced the bug yourself.

Has there been any real testing done to actually know if the timestamp is messed up? On a cursory inspection of those debug logs I posted about above it didn't seem as if the "last seen" timestamp reported from the USG or unifi switch was incorrect.

The way it is now the unifi integration in HA is absolutely useless for presence detection for people who don't have a "pure unifi" system (and my phone is again no longer being registered as home because the unifi_data file got re-updated to include it in the wireless category). And expecting people to switch out their existing hardware to "pure unifi" is at least a bit unreasonable due to the expense involved to solve this one bug.

I would be willing to put some effort into helping test any solutions to this problem on my system. That's why I offered to post logs or anything else I can do to help with this. Obviously I'm limited in being able to write any actual code to help mitigate this problem but please let me know if there is anything you could use to help fix it.

[Kane610]

@finity69x2 the quick fix here is to create an option to disable the wired bug management, it should probably behave like before then

[finity69x2]

@Kane610

And by "behaving like before" means that it will show as "home" for the "considered home" time after every HA restart? IOW, it will act just like it did before the bug fix in v103.6?

I'm not sure when this bug started but my phone hasn't reliably shown as "home" ever since I switched to the unifi gear in late august (long before v103.x). Ultimately the eventual fix for me was removing my phone's MAC from the unifi_data file (before I knew about this bug I had also tried running the controller software on three different machines). Then it would work for a day or so then go back to never showing as "home". Now I know that was because the unifi_data file got updated with data from the new controllers every time the controller IP changed..

if you would be interested in the backstory on this then here is the thread on my steps to lead me here.

https://community.home-assistant.io/t/continuing-my-unifi-presence-detection-woes/162102

I don't have any history on this bug aside from what i've read here (and the info in that thread) so I'm not sure exactly all that the "wired bug" entails. So I can't really say if that would be an acceptable "quick fix" or not.

[Kane610]

Yes, the assumptions in that thread are correct. So the solution to this issue would be to add an option to disable the wired_bug fix

[Kane610]

I will add an option to disable the wired bug remedy which should solve this issue. But Im in the middle of rewriting my automations which have been neglected for far too long so I just want to finish that first

[Mike1082]

@Kane610 Thank you. For what its worth, the Unifi device tracker has worked perfectly for me for a very long time. This issue began for me ONLY after recently doing maintenance on my network, and upgrading the controller software, USG, switch, and APs to the latest versions. Before that I had 0 issues.

[finity69x2]

@Mike1082

Thanks for dropping in and giving some info on this.

Do you know what the software versions were that you updated from that worked?

Is it possible to revert back to the old software versions to verify that it still works as expected even with the most recent updates to HA?

[Mike1082]

@finity69x2

Sorry I missed this... I do not know what the controller version was, and I have since upgraded hardware to a Unifi Dream Machine Pro. Unfortunately to my knowledge this integration is not yet compatible with the UDM Pro so I have not configured it yet to test.

[Kane610]

It only stores the state for it to survive restarts. It acts on the data as soon as it's there.

Haven't gotten to this issue for multiple reasons. It's still on my todo but my time for hass is on the low side right now

[GatoPharaoh]

thanks for the info!