Integrating with Homekit Controller finds no unpaired devices

grampadeal commented 4 years ago

The problem

Attempting to add Homekit Controller integration in the UI finds no unpaired devices

Environment

Home Assistant Core release with the issue: 0.114.3
Last working Home Assistant Core release (if known): N/A
Operating environment (OS/Container/Supervised/Core): HassIO On RPi3B+
Integration causing this issue: Homekit Controller
Link to integration documentation on our website: https://www.home-assistant.io/integrations/homekit_controller/

Problem-relevant `configuration.yaml`

n/a

Traceback/Error logs

no log entries

Additional information

I am running HA on a Raspberry Pi 3B+ connected to my network via ethernet. I don't have any VLANs set up that would prevent some devices from being able to contact others. My Philips Hue bridge (which is also on ethernet) is not detected, neither are my 2 Ecobee SmartThermostats. All 3 of these devices are detected (but not paired) in my iOS Homekit app.

probot-home-assistant[bot] commented 4 years ago

homekit_controller documentation homekit_controller source _{^{(message by IssueLinks)}}

Jc2k commented 4 years ago

When i run python -m netdisco from my home-assistant environment I see output like this:

Discovered devices:
homekit:
 {'host': '192.168.255.255',
  'hostname': 'Philips-hue.local.',
  'name': 'Philips hue - 11111',
  'port': 8080,
  'properties': {'c#': '21',
                 'ci': '2',
                 'ff': '1',
                 'id': 'xx:xx:xx:xx:xx:xx',
                 'md': 'BSB002',
                 'pv': '1.1',
                 's#': '1',
                 'sf': '0'}},

And 5 homekit devices are detected.

Does that command see the homekit devices when run from your raspberry pi? Can you share the output?

grampadeal commented 4 years ago

It doesn't look like there's a "homekit" entry on the output from that command. I had to modify it and run "python3 -m netdisco" since python wasn't the executable in my /usr/bin directory. Regardless, I uploaded the output. It found my Harmony Hub, my Plex server, my 2 Roku TVs and 1 standalone Roku, and the RPi running HA.

I originally had this in my configuration.yaml: discovery: enable:

homekit

But when I updated to 0.144.3, it told me that section was enabled by default, so I removed it. HA netdisco.txt

Edit: I don’t know if it’s relevant but I have both the Ecobee thermostats and the Hue bridges integrated with their HA integrations. I don’t think those would affect HomeKit integration, but I wanted to make note of that.

Jc2k commented 4 years ago

Yeah, it's going to be hard for me to help because i don't have the same environment as you - little things like python vs python3 are going to be common, unfortunately.

Regarding your other integrations. So Home Assistant in general prefers the "native" integration. I think at the moment a synchronous polling based cloud based integration is still chosen in preference to homekit_controller (lots of people have gotten emotional at being offered the choice of using an apple transport for their devices, so there is little desire to point out that homekit_controller is asynchronous, event based and local network only to those users). Devices that appear in this ticket and possibly others will intercept the HomeKit discovery data, and homekit_controller never gets a look-in.

If your device is not on that list, then there is no other filter. There is no API that lets Home Assistant work out that your native Ecobee integration is the same as the homekit device you are trying to pair.

However there are some reasons why thats none of this is relevant here:

I suspect you are already using the "+" button on the integrations page to initiate the pairing. This uses a different discovery mechanism and is not effected. It will see all unpaired devices. I'm pretty sure even with zeroconf turned off in your HA config.
The netdisco command you tried actually does not care about Home Assistant config or code at all, and it too couldn't find your devices.

netdisco is a standalone package that used to be used for zeroconf discovery in Home Assistant, but it has long since been replaced by the zeroconf integration. I think it is still kept around for other devices, but not HomeKit. This why you shouldn't put homekit in the discovery section any more. I use it here as its a good way of proving its not a fault of aiohomekit or the home assistant zeroconf integration because netdisco has the same problem and doesn't use either.

For example, you can install it outside of raspberry pi on any Linux box or mac with python3 and virtualenv support:

% python3 -m venv test-netdisco
% source test-netdisco/bin/activate
% pip install netdisco
Collecting netdisco
  Using cached https://files.pythonhosted.org/packages/bb/15/a74eeacc6afe050f7f1e68ff63598405fa7eb7d916ffe718c8b2d8fb0d11/netdisco-2.8.2-py3-none-any.whl
Collecting requests>=2.0 (from netdisco)
  Using cached https://files.pythonhosted.org/packages/45/1e/0c169c6a5381e241ba7404532c16a21d86ab872c9bed8bdcd4c423954103/requests-2.24.0-py2.py3-none-any.whl
Collecting zeroconf>=0.27.1 (from netdisco)
  Using cached https://files.pythonhosted.org/packages/f4/b4/52412d5d0feb98465d53a2c3b95a5b3916d62da70626adc3d898c9d0f682/zeroconf-0.28.1-py3-none-any.whl
Collecting chardet<4,>=3.0.2 (from requests>=2.0->netdisco)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests>=2.0->netdisco)
  Using cached https://files.pythonhosted.org/packages/5e/c4/6c4fe722df5343c33226f0b4e0bb042e4dc13483228b4718baf286f86d87/certifi-2020.6.20-py2.py3-none-any.whl
Collecting idna<3,>=2.5 (from requests>=2.0->netdisco)
  Using cached https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests>=2.0->netdisco)
  Using cached https://files.pythonhosted.org/packages/9f/f0/a391d1463ebb1b233795cabfc0ef38d3db4442339de68f847026199e69d7/urllib3-1.25.10-py2.py3-none-any.whl
Collecting ifaddr>=0.1.7 (from zeroconf>=0.27.1->netdisco)
  Using cached https://files.pythonhosted.org/packages/92/0f/a577a724c03982b800232713874e805c8fcc14f4a2c3060902ed20b50da8/ifaddr-0.1.7-py2.py3-none-any.whl
Installing collected packages: chardet, certifi, idna, urllib3, requests, ifaddr, zeroconf, netdisco
Successfully installed certifi-2020.6.20 chardet-3.0.4 idna-2.10 ifaddr-0.1.7 netdisco-2.8.2 requests-2.24.0 urllib3-1.25.10 zeroconf-0.28.1
WARNING: You are using pip version 19.2.3, however version 20.2.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
% python -m netdisco
Discovered devices:
homekit:

 {'host': '192.168.255.255',
  'hostname': 'Philips-hue.local.',
  'name': 'Philips hue - 11111',
  'port': 8080,
  'properties': {'c#': '21',
                 'ci': '2',
                 'ff': '1',
                 'id': 'xx:xx:xx:xx:xx:xx',
                 'md': 'BSB002',
                 'pv': '1.1',
                 's#': '1',
                 'sf': '0'}},

Discovered 1 devices

This is on a Mac and does not have access to my Home Assistant docker instance. I do have a Home Assistant dev instance on here, but it would have had to search my whole hard drive to find it.

So in my mind the output from your netdisco command shows that your devices aren't discoverable at all (paired or not) from the Raspberry Pi, and thats in a way that does not depend on your HA configuration and using a different code path to the one that you use inside HA. So we can rule out it being a configuration issue, and we can narrow it down to your environment (networking or the devices themselves), the python zeroconf library or 3 different projects are using the zeroconf library incorrectly.

The fact that the same problem occurs with 2 different vendors equipment, and I know Hue has one of the best HomeKit implementations, then I think its not likely to be the devices themselves that are at fault.

The scenarios we've got at the moment are:

The zeroconf UDP packets don't make it into the Home Assistant container running on HassIO (this could happen if your container is not running with host mode networking i guess)
There is a device on your network with a bad zeroconf implementation. It is interfering with discovery for devices with functional zeroconf. This has actually happened recently - e.g. https://github.com/home-assistant/core/pull/38851.

Can you try:

python3 -m netdisco raw

This will print out any raw zeroconf records that it did see.

I imagine the container doesn't contain tcpdump but if it did i'd suggest something like:

tcpdump -ni en1 port 5353 -X

to see if it saw your phillips hue adverstising itself by zeroconf.

grampadeal commented 4 years ago

Thanks for the lengthy/detailed response! Indeed, the native Ecobee integration is more feature-rich than the Homekit integration, but the downside is the reliance on Ecobee's cloud servers. Since Homekit grants local control over the thermostats, I wanted to have that option in case Ecobee's servers go down.

So I did some research and it looks like my Unifi networking equipment can sometimes interfere with network discovery of devices, depending on configuration. I have a Unifi USG, 24 port switch, and AP. Other brands of networking equipment could also cause the issue I was seeing.

Anyway, I went into my controller and disabled IGMP snooping on my LAN and that cleared up the issue. It looks like that IGMP snooping can interfere with multicast traffic on networks, where the switch examines multicast traffic and decides which devices on the network need to have that traffic forwarded. If the switch doesn't think a particular device needs to have the multicast request forwarded to it, it'll simply drop the request.

When I disabled this, the next time I ran python3 -m netdisco, all homekit devices were detected immediately. Possibly related to this (but not related to Homekit or Home Assistant), I have seen periodic, brief disconnects with my Hue bridge from the network where devices will, for short periods, be unavailable. I am going to keep an eye on this, but I suspect IGMP snooping to be responsible for this.

Here's a related reddit post where the user had the same issue I had with the same solution: https://www.reddit.com/r/homeassistant/comments/dm0wim/til_home_automation_and_network/

Regardless, this issue can be closed since it was related to my local network configuration.

TL;DR: If Homekit Controller can't find any devices (when clicking the "+" button in the Integrations page), check to see if IGMP snooping is enabled on your router/switch and disable it.

Jc2k commented 4 years ago

Totally agree with the sentiment about preferring local over cloud! Hopefully when homekit_controller is more mature we will be able to get more people on board :-)

I'm super glad you were able to figure this out. I have UniFi gear myself, and weirdly for the VLAN HA is on IGMP snooping is enabled and i've not had any trouble. Wonder at what point it becomes problematic...

Now that your Ecobee devices are working, I imagine you'll be interested in this issue. Also i'm a Hue user too - you might be interested in this when it is merged.

Glad it is working for you now!

home-assistant / core