home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.73k stars 29.99k forks source link

homekit_controller does not work with devices that have duplicate serial numbers #34121

Closed elektron303 closed 2 years ago

elektron303 commented 4 years ago

The problem

Platform homekit_controller does not generate unique IDs. Not all sensors show up.

I've successfully added my heater with homekit controller, but only the controller itself and one sensor shows up instead of 12 sensors. Integration in iOS works without problems.

Environment

Problem-relevant configuration.yaml

Traceback/Error logs

2020-04-12 23:07:56 INFO (SyncWorker_8) [homeassistant.loader] Loaded climate from homeassistant.components.climate
2020-04-12 23:07:56 INFO (MainThread) [homeassistant.setup] Setting up climate
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=climate, service=turn_on>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=climate, service=turn_off>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=climate, service=set_hvac_mode>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=climate, service=set_preset_mode>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=climate, service=set_aux_heat>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=climate, service=set_temperature>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=climate, service=set_humidity>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=climate, service=set_fan_mode>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: domain=climate, service=set_swing_mode>
2020-04-12 23:07:56 INFO (MainThread) [homeassistant.setup] Setup of domain climate took 0.0 seconds.
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=climate>
2020-04-12 23:07:56 INFO (MainThread) [homeassistant.components.climate] Setting up climate.homekit_controller
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 5, 'type': 'event', 'event': {'event_type': 'service_registered', 'data': {'domain': 'climate', 'service': 'turn_on'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 545772, tzinfo=<UTC>), 'context': {'id': '547726b079434e8f8dacb635f7aef420', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 5, 'type': 'event', 'event': {'event_type': 'service_registered', 'data': {'domain': 'climate', 'service': 'turn_off'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 547172, tzinfo=<UTC>), 'context': {'id': '2f677e9449614b40bbc43188ff2affd5', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 5, 'type': 'event', 'event': {'event_type': 'service_registered', 'data': {'domain': 'climate', 'service': 'set_hvac_mode'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 548767, tzinfo=<UTC>), 'context': {'id': '3864f4dcc4d64528985a02c6fd0e46d8', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 5, 'type': 'event', 'event': {'event_type': 'service_registered', 'data': {'domain': 'climate', 'service': 'set_preset_mode'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 550305, tzinfo=<UTC>), 'context': {'id': '1e1232e7da0741d5a2f77b69a07a806f', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 5, 'type': 'event', 'event': {'event_type': 'service_registered', 'data': {'domain': 'climate', 'service': 'set_aux_heat'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 551827, tzinfo=<UTC>), 'context': {'id': 'dae04a18eb6d4c87bfe7f4c26beeb8a5', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 5, 'type': 'event', 'event': {'event_type': 'service_registered', 'data': {'domain': 'climate', 'service': 'set_temperature'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 552945, tzinfo=<UTC>), 'context': {'id': 'cc07cb2f147a4d15a43ee544931724e4', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 5, 'type': 'event', 'event': {'event_type': 'service_registered', 'data': {'domain': 'climate', 'service': 'set_humidity'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 554480, tzinfo=<UTC>), 'context': {'id': '2bc12ee271674bb29575ab83409d427c', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 5, 'type': 'event', 'event': {'event_type': 'service_registered', 'data': {'domain': 'climate', 'service': 'set_fan_mode'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 556283, tzinfo=<UTC>), 'context': {'id': 'a997098e8e0d4490b8e6664a0d58ea5b', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 5, 'type': 'event', 'event': {'event_type': 'service_registered', 'data': {'domain': 'climate', 'service': 'set_swing_mode'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 557914, tzinfo=<UTC>), 'context': {'id': 'b77ab3f4193843a2be0219d039ace447', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.components.websocket_api.http.connection.140422955157520] Sending {'id': 3, 'type': 'event', 'event': {'event_type': 'component_loaded', 'data': {'component': 'climate'}, 'origin': 'LOCAL', 'time_fired': datetime.datetime(2020, 4, 12, 23, 7, 56, 559964, tzinfo=<UTC>), 'context': {'id': 'f00419621f834468aed82b66ed055263', 'parent_id': None, 'user_id': None}}}
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=create, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 INFO (MainThread) [homeassistant.helpers.entity_registry] Registered new climate.homekit_controller entity: climate.wohnzimmer_og
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=create, entity_id=climate.wohnzimmer_og>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=climate.wohnzimmer_og, old_state=None, new_state=<state climate.wohnzimmer_og=off; hvac_modes=['off', 'heat', 'cool', 'heat_cool'], min_temp=5, max_temp=30, current_temperature=24.4, temperature=10, hvac_action=idle, friendly_name=Wohnzimmer (OG), supported_features=1 @ 2020-04-13T01:07:56.617726+02:00>>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event device_registry_updated[L]: action=update, device_id=fa903c841d184780a7d809e08a24981d>
2020-04-12 23:07:56 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event entity_registry_updated[L]: action=update, entity_id=climate.wohnzimmer_og, changes=['original_name']>
2020-04-12 23:07:56 ERROR (MainThread) [homeassistant.components.climate] Entity id already exists - ignoring: climate.wohnzimmer_og. Platform homekit_controller does not generate unique IDs

Additional information

probot-home-assistant[bot] commented 4 years ago

Hey there @Jc2k, mind taking a look at this issue as its been labeled with a integration (homekit_controller) you are listed as a codeowner for? Thanks!

Jc2k commented 4 years ago

It does generate unique ids but only if the devices have unique serial numbers. This tends not to be a problem for Apple sanctioned firmware’s (so far) but custom firmwares or home bridge setups especially seem to sometimes set it to a static string like “000000”.

I can verify this with your .storage/homekit_controller-entity-map file if you like.

There is a ticket about this and it proposes a fix - long term we can mix in the pairing id and accessory id into the unique id to work for these devices. The aid isn’t very unique but is when mixed with the pairing id it’s fine. It’s not stable between pairings is annoying.

But it’s complicated - unique is is not meant to change and changing this will disrupt things for every user who got it working in the past 2 years.

elektron303 commented 4 years ago

Hello Jc2k!

I've attached the homekit_controller-entity-map file. Can I manually change the serial numbers or is there another workaround for this problem?

homekit_controller-entity-map.txt

Thank you!

Jc2k commented 4 years ago

What kind of device are you pairing? Is it custom firmware? Some kind of software bridge?

elektron303 commented 4 years ago

It is a "smart heater" by the company "Controme" (https://shop.controme.com/produkt/smart-home-heizungssteuerung-mini-server/) Basically they use a Raspi with python scripts and what it seems like a poorly implemented version of Nick Farinas Homebridge: https://github.com/homebridge/homebridge

Jc2k commented 4 years ago

That’s unfortunate - you would need to change the home bridge plugin to make it work. If you have shell access you might be able to but I don’t know how they handle upgrades?

elektron303 commented 4 years ago

I will contact the manufacturer to fix it. If I got this right the problem is that the sensors don't have unique IDs. Instead of unique IDs they have "value": "Default-SerialNumber" as serial number? Is that correct? Is there any known workaround for this problem? Can I edit the homekit_controller-entity-map manually maybe?

Jc2k commented 4 years ago

Basically, yes.

To be more explicit, Home Assistant's implementation of HomeKit relies on each accessory (not just the bridge) having a unique serial number.

As you have seen, in the case of this device the root bridge accessory has a serial number (set to the device id, which is a Mac address like ID that is supposed to change every time the HomeKit pairing is changed, so i assume its not the real serial number). Then there are 12 accessories with this "Default-SerialNumber" value.

Home Assistant does not care what the serial number is - just that it is unique and that it is stable between reboots. So using the mac-like serial number from the bridge and then appending the "aid" field from each accessory would be fine for Home Assistant. Obviously it would look like a more polished commercial product if they populated the real serial numbers.

When this happened last time the manufacturer refused to help. If they do, please stress that homekit_controller is a vounteer effort and fixing this would be a breaking change for many of our users at the moment. Where as changing their homebridge plugin would be a non-breaking change for them. So I hope they will be more accomodating...

elektron303 commented 4 years ago

Thank you Jc2k! I will close this issue for now. If there will be an update I will let you know!

ioannispelelis commented 3 years ago

I have the same issue and the supplier of the homekit bridge is unwilling to make any change. As such i have only one door/window sensor that appear instead of 5 I guess there is no way around it?

Jc2k commented 3 years ago

No sorry. Very surprised they passed certification with the serial field being incorrectly set. It's a mandatory field. I guess technically the spec only says the serial number field must exist and contain at least 1 character, but I still doubt Apple would certify "000000" or similar.

(If they aren't actually certified and they are using the HomeKit branding they might be in violation of trademark law fwiw).

ioannispelelis commented 3 years ago

Unfortunately they say that the provide their HomeKit VM to control the alarm system with Apple Home App. If i use it some other way that what they intended (eg interfacing it with HomeAssistant instead of Apple Home App) they do not provide support... I guess there is not much else i can do

Jc2k commented 3 years ago

Who are the manufacturer btw?

ioannispelelis commented 3 years ago

It if the french ISP provider "Free" They have a router box called "Freebox Delta" that has an integrated alarm system. Actually Freebox exists as an integration in HomeAssistant but does not support the alarm system part (at least for now) Free recently added the possibility to add a VM on the router that will interface with Apple Home (this is what i am using to interface with HomeAssistant instead) https://www.smarthome.freebox.fr/article/178-tout-savoir-sur-homebridge

It already works i can enable/disalbe alarme system. But i can see only one sensor status (instead of 5 since the other 4 are duplicates)

Jc2k commented 3 years ago

Ah, homebridge. Every time this comes up its homebridge. It's not a certified implementation.

It looks like the code is here? https://github.com/fbx/homebridge-freebox-home Is that right?

ioannispelelis commented 3 years ago

I guess this must be it yes. What the ISP has put in place and i used is a "self packaged" small linux VM (that i guess must download and install this github project)

Jc2k commented 3 years ago

Interesting! It's quite a "heath robinson" setup they've got going on, more so than i'd expect from a company. That project is a meta project that provides 2 parts. The first is a homebridge installer and config generator. It uses 4 "standard" homebridge plugins:

The project can't itself actually set the serial numbers that appear to HomeKit itself if the plugins don't expose a mechanism to do that. Here is where the motion sensor is configured:

https://github.com/Que20/homebridge-http-motion-sensor/blob/master/index.js#L75

                informationService
                        .setCharacteristic(Characteristic.Manufacturer, "ContactSensor")
                        .setCharacteristic(Characteristic.Model, "FrontDoor")
                        .setCharacteristic(Characteristic.SerialNumber, "Version 1.0.3");

I would argue that all 3 fields are set incorrectly (its meant to be a motion sensor, it has no idea where the sensor is but you clearly don't have 5 pointed at the front door, and that serial number is a version number). Even it "works" with iOS, it's a pretty embarassing position for a company to take that this isn't a bug on their part, even if it is a minor one. If i was making a HomeKit bridge for my companies product i would at least get the manufacturer right!

It also provides a http proxy. This proxy adapts calls from what these off-the-shelf plugins make to the format the freebox expects. Which means that any interaction with your devices actually looks like this:

[home-assistant / iOS] -> [homebridge with http plugins] -> [their proxy] -> [the "Freebox" API]

The API works by polling AIUI, even though HomeKit supports push notifications. So the homebridge http plugins will generate a request once a second. This will be translated by the proxy into a valid request for the actual freebox API.

In general it is hard for me to support things like this because homebridge is not a certified implementation and a lot of the setups involve lots of weird stuff like this. And when you mash together so many layers like this security problems are inevitable. Meanwhile, I have to learn about 3 different services to understand whats happening in your problem 😱

Speaking of which. It looks like those proxy API's are unauthenticated, and it looks like it is configured to listen on it's hosts interface (it encourages you to look at urls via your host's ip address). This concerns me about how secure your home security system is. Reading the code, there is an endpoint to disarm your alarm. It looks like they have tried to block access to the disarm endpoint by checking the IP that the connection is to:

    this.checkUnauthorizedRequest = function(request) {
        if (request.headers.host != 'localhost:'+this.port) {
            request.status(401)
            request.send('unauthorized')
            return
        }
    }

In most HTTP frameworks headers are set by the client and should not be trusted. A proxy might overide them, but my expecation on reading this code is that if you go to (replace 1.2.3.4 with the IP of the VM):

curl http://1.2.3.4:8888/version

You should see a version number. If you can find the IP and port where this works I believe this will then also work:

curl -H "Host: localhost:8888" http://1.2.3.4:8888/alarm/off

From a cursory reading of the code (not written in my usual language or framework) i believe anyone on your wifi can now disarm your home security system without a passcode. (It's possible this is mitigated elsewhere in the stack of course).

ioannispelelis commented 3 years ago

I tried the curl commands you were speaking off and i am just getting a response "Cannot GET /version" or "Cannot GET /alarm/off" I don't know if they have not mitigated the risk you are talking about by the fact that the VM is running on the very router where the alarm system is integrated. So maybe the calls go directly from the alarm to the VM and back? without transiting over the lan/wifi network?

About the contact and motion sensors' serial number issue what do you think are the options?

Jc2k commented 3 years ago

Can you run the commands with the verbose flag and post the output? (-v). I want to verify what is making the response. There are so many different HTTP servers involved in this project it would be easy to test the wrong one and think you are safe when you are not. The homebridge part of this would have to directly be exposed to your network for mDNS to work properly which is why i'm not convinced that it is.

It looks like you could run homebridge directly and install the plugins directly. There is some help on doing this here. Then you could modify all the plugins i modified to have "serial numbers". They don't have to be real, just unique. I'd probably use something like the URL it is configured to use as the serial as from what i can tell that should be unique for each motion sensor (see here where the freebox generates the unique url for each sensor). I believe you could change this line to be something like:

                informationService
                        .setCharacteristic(Characteristic.Manufacturer, "ContactSensor")
                        .setCharacteristic(Characteristic.Model, "FrontDoor")
                        .setCharacteristic(Characteristic.SerialNumber, this.statusURL);

If i knew JS a bit more i'd probably make a hash so that it had a fixed and predictable length just in case the url itself was too long to be a valid serial number.

I don't use homebridge so these are just hunches. You are going to need to understand enough JS to install and patch the homebridge-http-motion-sensor plugin to make this work...

ioannispelelis commented 3 years ago

This is the verbose version of the command.

curl -v http://1.2.3.4:8888/version
*   Trying 1.2.3.4...
* TCP_NODELAY set
* Connected to 1.2.3.4 (1.2.3.4) port 8888 (#0)
> GET /version HTTP/1.1
> Host: 1.2.3.4:8888
> User-Agent: curl/7.63.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< X-Powered-By: Express
< Content-Security-Policy: default-src 'self'
< X-Content-Type-Options: nosniff
< Content-Type: text/html; charset=utf-8
< Content-Length: 146
< Date: Mon, 23 Nov 2020 17:21:58 GMT
< Connection: keep-alive
<
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /version</pre>
</body>
</html>
* Connection #0 to host 1.2.3.4 left intact

Hmmm guess i would have to learn about HomeBridge and learn Javascript for what you are talking about... it is currently a little out of my field of expertise :) I will try to dig a little when i have time I guess the most ideal solution and more secure way would be if the Home Assistant official Freebox integration could support in the future the Freebox API for the alarm system. Who knows maybe one day it will be :)

Jc2k commented 3 years ago

Can you try again but add 'api/' to the routes - so:

curl -v http://1.2.3.4:8888/api/version
curl -v -H 'Host: localhost:8888'  http://1.2.3.4:8888/api/alarm/off

Not used to "Express" so missed that last time.

ioannispelelis commented 3 years ago

Here are the two commands as requested:

curl -v http://1.2.3.4:8888/api/version
*   Trying 1.2.3.4...
* TCP_NODELAY set
* Connected to 1.2.3.4 (1.2.3.4) port 8888 (#0)
> GET /api/version HTTP/1.1
> Host: 1.2.3.4:8888
> User-Agent: curl/7.63.0
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Content-Type: text/html; charset=utf-8
< Content-Length: 5
< ETag: W/"5-oORWSp3E0j+NuNvVXVkJkQ2rtUo"
< Date: Mon, 23 Nov 2020 18:46:49 GMT
< Connection: keep-alive
<
da147* Connection #0 to host 1.2.3.4 left intact
curl -v -H 'Host: localhost:8888'  http://1.2.3.4:8888/api/alarm/off
* Closing connection -1
curl: (3) URL using bad/illegal format or missing URL
*   Trying 1.2.3.4...
* TCP_NODELAY set
* Connected to 1.2.3.4 (1.2.3.4) port 8888 (#0)
> GET /api/alarm/off HTTP/1.1
> Host: 1.2.3.4:8888
> User-Agent: curl/7.63.0
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< X-Powered-By: Express
< Content-Security-Policy: default-src 'self'
< X-Content-Type-Options: nosniff
< Content-Type: text/html; charset=utf-8
< Content-Length: 1239
< Date: Mon, 23 Nov 2020 19:00:00 GMT
< Connection: keep-alive
<
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>TypeError: request.status is not a function<br> &nbsp; &nbsp;at module.exports.checkUnauthorizedRequest (/homebridge-freebox-home/src/routes/Routes.js:30:21)<br> &nbsp; &nbsp;at /homebridge-freebox-home/src/routes/Routes.js:90:18<br> &nbsp; &nbsp;at Layer.handle [as handle_request] (/homebridge-freebox-home/node_modules/express/lib/router/layer.js:95:5)<br> &nbsp; &nbsp;at next (/homebridge-freebox-home/node_modules/express/lib/router/route.js:137:13)<br> &nbsp; &nbsp;at Route.dispatch (/homebridge-freebox-home/node_modules/express/lib/router/route.js:112:3)<br> &nbsp; &nbsp;at Layer.handle [as handle_request] (/homebridge-freebox-home/node_modules/express/lib/router/layer.js:95:5)<br> &nbsp; &nbsp;at /homebridge-freebox-home/node_modules/express/lib/router/index.js:281:22<br> &nbsp; &nbsp;at Function.process_params (/homebridge-freebox-home/node_modules/express/lib/router/index.js:335:12)<br> &nbsp; &nbsp;at next (/homebridge-freebox-home/node_modules/express/lib/router/index.js:275:10)<br> &nbsp; &nbsp;at Function.handle (/homebridge-freebox-home/node_modules/express/lib/router/index.js:174:3)</pre>
</body>
</html>
* Connection #0 to host 1.2.3.4 left intact
Jc2k commented 3 years ago

I wonder why you got 'curl: (3) URL using bad/illegal format or missing URL' for the 2nd example url (i don't). I've found a few other issues so i've reached out to the developer privately.

Jc2k commented 3 years ago

@ioannispelelis the maintainer suggested that the homebridge code is considered experimental, labelled as experimental and is not and should not be used in production. If I understood the reply correctly they seemed to suggest that they don't plan to maintain it going forward (a company decision).

They are aware of the loophole I pointed out above. I also found another endpoint which would allow me (if i had access to your wifi) to forcefully unpair your iOS device and clear the way for me to pair my own iPhone in its place. I believe the default pairing code is the same for anyone with that VM, so the normal iOS countermeasures would be ineffective. This would allow me (depending on which devices you have) to view any cameras you had and disarm your alarm at will.

Apparently they will be updating the github repo shortly to explain that the project is EOL.

Personally i would remove the HomeKit integration VM from my network.

ioannispelelis commented 3 years ago

@Jc2k thanks for checking so thoroughly Actually i have no iPhone device paired with the homebridge VM. I just downloaded the company provided VM to my router. And once the VM was ready and installed instead of pairing it with Apple Home App on an iPhone i paired to HomeAssistant If I paired it with an iPhone during the initial setup then the Homebridge could never be paired again with HomeAssistant and I had to scratch the VM and redownload and let the setup run its course again. Now the VM i got running is paired with HomeAssistant only and an Apple iPhone device cannot pair with it anymore

I wonder if the code you found here https://github.com/fbx/homebridge-freebox-home is not a different project Because from what I can understand from the project they are talking about installing the components to interface on an existing homebridge intance. The "official" company solution (explained here: https://www.smarthome.freebox.fr/article/178-tout-savoir-sur-homebridge) downloads a complete ready to use VM only for controlling the alarm system and interfacing with Apple Home (that is what I used but instead of interfacing with Apple Home I interfaced with Homekit integration of my HomeAssistant)

So maybe the github project is EOL but now the official company VM? I mean they launched it pretty recently it is weird to abandon it so soon..

Jc2k commented 3 years ago

The repository i pointed to is in the GitHub of the company that makes the freebox. The guy i spoke to works for them, on stuff like this, his resume is on his website. The code running on your VM is that GitHub project because you made it do a 500 error and it gave a traceback. That traceback gave clues about what was running. Look at the paths - /homebridge-freebox-home/src/routes/Routes.js. The line numbers and the files match that GitHub project. Even if there is a secret branch they have internally it is still fundamentally based on the code on their github, with the security issues i pointed out. The fact that it gave up its file structure was actually one of the issues i raised because a production grade deployment in a VM should just not do that. It is close enough that you should be concerned about running it in production.

The guy even told me what the future direction for the project was if it hadn't been cancelled (it wasn't homebridge based), and that the company would be pulling away from some of their smart home projects going forward, not just this one. It sounded like it was a recent development.

ioannispelelis commented 3 years ago

Oh Ok thanks for your all time and effort. I guess i will have to remove this VM eventually if it will be EOL and a security risk.. Hopefully they wont stop all their alarm system from working standalone... Otherwise i willl have to think of another solution. Thanks again!

alex4988 commented 3 years ago

Hi Guys, i have same issue with similar environment(only unique difference HA version is 0.118.4).

Platform homekit_controller does not generate unique IDs. ID homekit-none-8 already exists - ignoring cover.camera

How can i help(you need other logs or further information on my configuration)?

Thanks a lot

Jc2k commented 3 years ago

Are you using homebridge?

alex4988 commented 3 years ago

Yes i'm using homebridge

Jc2k commented 3 years ago

The plugins you are using don't correctly populate the serial number field. This is a mandatory field in HomeKit. Unfortunately while the iOS client tolerates it being missing or wrong HomeAssistant relies on it to establish a persistent mapping between the device and HA internal state. Right now the only way forward is to fix the plug-in which I can't help you with.

alex4988 commented 3 years ago

The plugins you are using don't correctly populate the serial number field. This is a mandatory field in HomeKit. Unfortunately while the iOS client tolerates it being missing or wrong HomeAssistant relies on it to establish a persistent mapping between the device and HA internal state. Right now the only way forward is to fix the plug-in which I can't help you with.

I try to import in home assistant all my window covering that are all controlled in openHAB. I have all my home automations in home assistant except my window covering, so by importing these devices in Home Assistant I will be able to controll all my devices in home assistant.

I try to use "homeKit" but in my homekit_controller-entity-map file I have:

`

                    "aid": 339449962,
                    "services": [
                        {
                            "iid": 1,
                            "type": "0000003E-0000-1000-8000-0026BB765291",
                            "characteristics": [
                                {
                                    "iid": 2,
                                    "type": "00000014-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pw"
                                    ],
                                    "format": "bool",
                                    "ev": false,
                                    "description": "identifies the accessory"
                                },
                                {
                                    "iid": 3,
                                    "type": "00000020-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "manufacturer",
                                    "value": "none",
                                    "maxLen": 255
                                },
                                {
                                    "iid": 4,
                                    "type": "00000021-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "model",
                                    "value": "none",
                                    "maxLen": 255
                                },
                                {
                                    "iid": 5,
                                    "type": "00000023-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "name",
                                    "value": "Studio",
                                    "maxLen": 255
                                },
                                {
                                    "iid": 6,
                                    "type": "00000030-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "serial number",
                                    "value": “none”,
                                    "maxLen": 255
                                },
                                {
                                    "iid": 7,
                                    "type": "00000052-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "firmware revision",
                                    "value": "none",
                                    "maxLen": 255
                                }
                            ],
                            "stype": "accessory-information"
                        },
                        {
                            "iid": 8,
                            "type": "0000008C-0000-1000-8000-0026BB765291",
                            "characteristics": [
                                {
                                    "iid": 9,
                                    "type": "0000007C-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr",
                                        "pw",
                                        "ev"
                                    ],
                                    "format": "int",
                                    "ev": false,
                                    "description": "target position",
                                    "value": 0,
                                    "minValue": 0,
                                    "maxValue": 100,
                                   "minStep": 1,
                                    "unit": "%"
                                },
                                {
                                    "iid": 10,
                                    "type": "0000006D-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr",
                                        "ev"
                                    ],
                                    "format": "int",
                                    "ev": false,
                                    "description": "current position",
                                    "value": 0,
                                    "minValue": 0,
                                    "maxValue": 100,
                                    "minStep": 1,
                                    "unit": "%"
                                },
                                {
                                    "iid": 11,
                                    "type": "00000072-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr",
                                        "ev"
                                    ],
                                    "format": "int",
                                    "ev": false,
                                    "description": "Position state",
                                    "value": 2,
                                    "minValue": 0,
                                    "maxValue": 2,
                                    "minStep": 1
                                }
                            ],
                            "stype": "window-covering"
                        }
                    ]
                },
                {
                    "aid": 2011083194,
                    "services": [
                        {
                            "iid": 1,
                            "type": "0000003E-0000-1000-8000-0026BB765291",
                            "characteristics": [
                                {
                                    "iid": 2,
                                    "type": "00000014-0000-1000-8000-0026BB765291",
                                   "perms": [
                                        "pw"
                                    ],
                                    "format": "bool",
                                    "ev": false,
                                    "description": "identifies the accessory"
                                },
                                {
                                    "iid": 3,
                                    "type": "00000020-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "manufacturer",
                                    "value": "none",
                                    "maxLen": 255
                                },
                                {
                                    "iid": 4,
                                    "type": "00000021-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "model",
                                    "value": "none",
                                    "maxLen": 255
                                },
                                {
                                    "iid": 5,
                                    "type": "00000023-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "name",
                                    "value": "Camera",
                                    "maxLen": 255
                                },
                                {
                                    "iid": 6,
                                    "type": "00000030-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "serial number",
                                    "value": “none”,
                                    "maxLen": 255
                                },
                                {
                                    "iid": 7,
                                    "type": "00000052-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr"
                                    ],
                                    "format": "string",
                                    "ev": false,
                                    "description": "firmware revision",
                                    "value": "none",
                                    "maxLen": 255
                                }
                            ],
                            "stype": "accessory-information"
                        },
                        {
                            "iid": 8,
                            "type": "0000008C-0000-1000-8000-0026BB765291",
                            "characteristics": [
                                {
                                    "iid": 9,
                                    "type": "0000007C-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr",
                                        "pw",
                                        "ev"
                                    ],
                                    "format": "int",
                                    "ev": false,
                                    "description": "target position",
                                    "value": 0,
                                    "minValue": 0,
                                    "maxValue": 100,
                                    "minStep": 1,
                                    "unit": "%"
                                },
                                {
                                    "iid": 10,
                                    "type": "0000006D-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr",
                                        "ev"
                                    ],
                                    "format": "int",
                                    "ev": false,
                                    "description": "current position",
                                    "value": 0,
                                    "minValue": 0,
                                    "maxValue": 100,
                                    "minStep": 1,
                                    "unit": "%"
                                },
                                {
                                    "iid": 11,
                                    "type": "00000072-0000-1000-8000-0026BB765291",
                                    "perms": [
                                        "pr",
                                        "ev"
                                    ],
                                    "format": "int",
                                    "ev": false,
                                    "description": "Position state",
                                    "value": 2,
                                    "minValue": 0,
                                    "maxValue": 2,
                                    "minStep": 1
                                }
                            ],
                            "stype": "window-covering"
                        }
                    ]
                }

` As you can see, the serail numbers are all valued with "none". What can I do? Do you have any suggestions to fix it?

Thanks

Jc2k commented 3 years ago

If openhab does not put a stable unique identifier in the serial like a valid certified accessory then it won't work, and there's not really anything I can do.

yuqianma commented 2 years ago

It does generate unique ids but only if the devices have unique serial numbers. This tends not to be a problem for Apple sanctioned firmware’s (so far) but custom firmwares or home bridge setups especially seem to sometimes set it to a static string like “000000”.

I can verify this with your .storage/homekit_controller-entity-map file if you like.

There is a ticket about this and it proposes a fix - long term we can mix in the pairing id and accessory id into the unique id to work for these devices. The aid isn’t very unique but is when mixed with the pairing id it’s fine. It’s not stable between pairings is annoying.

But it’s complicated - unique is is not meant to change and changing this will disrupt things for every user who got it working in the past 2 years.

Why not set an option to turn on the new id logic? Enable it for the brand new install but keep it be false for the current install. Since Apple accepts it, staying the same behavior as Apple should be convenient.

Jc2k commented 2 years ago

No, I'm against options here. We should make it work automatically. We know what the broken devices look like. It works for Apple without an option.

It's just someone having time to do the work and test it.

BeatYabad commented 2 years ago

This is also happening with the sensors used by Honeywell T9 and T10 thermostats.

Jc2k commented 2 years ago

Can you post your homekit_controller-entity-map so I can verify its caused by the same thing.

BeatYabad commented 2 years ago

Can you post your homekit_controller-entity-map so I can verify its caused by the same thing.

I have 3 Honeywell T10s with multiple sensors for each. It actually looks like the first sensor for each one is given the same idea. Same goes for 2nd and 3rd. So I'm not sure it is identical to this issue or not. Here's the file: homekit_controller-entity-map.txt

Jc2k commented 2 years ago

If you look at all occurrences of 00000030-0000-1000-8000-0026BB765291 do you see dupes? I'll look myself in the morning but if you do it's the same issue. (On phone now and formatting is screwy).

BeatYabad commented 2 years ago

I'm a bit of a newb with HA and I see multiple attributes with that type value. For example, 2 entities have this value setting: { "iid": 6, "type": "00000030-0000-1000-8000-0026BB765291", "format": "string", "value": "33215000300000", "perms": [ "pr" ], "ev": false },

Jc2k commented 2 years ago

Ah ok. It's ok, I'll look in the morning. This is a homekit thing rather than a HA thing. The uuid I said to look for is for serial numbers. HA needs devices to be uniquely identifiable. 33215000300000 is the serial number of one of your devices. But if you see 33215000300000 multiple times that means that homekit can't actually persistently and reliably differentiate between 2 or more devices. Which is what this bug is about.

Where the duplicates are part of the same pairing we should be able to easily and automatically fallback to a less reliable mechanism for these semi-broken devices.

BeatYabad commented 2 years ago

Thanks. FWIW, I was able to add all thermostats and their sensors to HomeKit without issue.

Sent from my iPhone

On Jan 16, 2022, at 5:20 PM, Jc2k @.***> wrote:



Ah ok. It's ok, I'll look in the morning. This is a homekit thing rather than a HA thing. The uuid I said to look for is for serial numbers. HA needs devices to be uniquely identifiable. 33215000300000 is the serial number of one of your devices. But if you see 33215000300000 multiple times that means that homekit can't actually persistently and reliably differentiate between 2 or more devices. Which is what this bug is about.

Where the duplicates are part of the same pairing we should be able to easily and automatically fallback to a less reliable mechanism for these semi-broken devices.

— Reply to this email directly, view it on GitHubhttps://github.com/home-assistant/core/issues/34121#issuecomment-1013964194, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABSTRUMHEUEAYGLJ2PLYPJTUWNAEFANCNFSM4MGTE53A. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you commented.Message ID: @.***>

BeatYabad commented 2 years ago

I found a work-around for the Honeywell T9 and T10 thermostats' sensors. It looks like the serial number is generated by the Thermostat when the sensor is added to it. The value is incremented with each sensor and the number is not reused. So I have 3 thermostats with 3 sensors each. I had to remove and readd six sensors over and over to get their numbers to increment until I had 1-9.

Jc2k commented 2 years ago

Re-closing in favour of https://github.com/home-assistant/core/issues/60667. Congrats on finding a workaround.

Rahulsharma0810 commented 2 years ago

I will come back to HA once the issue is fixed, my major accessories are giving the same error.

kevinhaas commented 3 months ago

I still have this issue, but I added my Honeywell stuff before https://github.com/home-assistant/core/pull/80064 was merged.

If I delete and re-add my Honeywell stuff will this issue be fixed then?