Open ebaauw opened 2 years ago
Hey, I am new to open source and I am finding it really difficult as to how to start doing anything. I just came across your project, could you guide/explain as to what should I do because I am interested in contributing to open source, so taking your project as a start can you help me contribute?
@HARDIK-TSH1392 better start with something far less technically complex.
Wow. That's a lot of work you have to do. So much code and I'm only using it for two devices. And I don't have enough programming skills to contribute anything essential. If you're ever in Frankfurt am Main, your dinner's on me. But maybe you have some better ideas how we can support you.
@ebaauw thank you for your job
my whole smart home is based on you plugin
can’t wait for your next project
Hi Erik, wise decision! 🙏🏻 I think this will take out some complexity and better focus on both platforms. When do you suspect it will stable for usage? Will there be a possibility to migrate?
Homebridge deCONZ should be pretty stable by now.
I cannot opaquely move accessories from Homebridge Hue to Homebridge deCONZ or Homebridge Hue2, without HomeKit noticing. In theory this would only be possible if all plugins use the same Homebridge installation (without child bridges). As Homebridge deCONZ and Homebridge Hue2 generate the same accessory UUID as Homebridge Hue, HomeKit wouldn't care which plugin handles the accessory (and doesn't even know), as long as it's exposed by the same HomeKit bridge (HAP server). However, only one plugin can expose that accessory, or you would see the dreaded "cannot add another accessory with the same UUID" message.
Homebridge Hue still uses the (ancient) static platform accessory model, where all accessories need to be declared to Homebridge, before it starts the HAP server. Homebridge deCONZ and Homebridge Hue2 use the (newer) dynamic platform accessory model, where a plugin can add (and remove) accessories at runtime, after Homebridge has started the HAP server. So I would need to restart Homebridge for Homebridge Hue to no longer expose an accessory, but Homebridge deCONZ and Homebridge Hue2, can only re-create that accessory, after the HAP server has started. This would leave a small window where the accessory isn't exposed, causing HomeKit to remove the accessory, and to treat the re-created accessory as a new one. I did have a hack in homebridge-lib, to delay the startup of Homebridge from a dynamic platform plugin, but that's too complex to share and no longer works with child bridges. I discussed with the team to include this feature in Homebridge, but it was deemed too dangerous, as too many users wouldn't understand why Homebridge wouldn't be visible in HomeKit, even though it is running.
To make a long story short: migration consists of removing the accessories by Homebridge Hue from HomeKit, and re-creating them by Homebridge deCONZ or Homebridge Hue2, losing any associations to HomeKit rooms, groups, scenes, automations. This sucks, but then, so does life. To mitigate the impact, I support manual migration per accessory, so you don't have to handle all the re-configuration at once. And it also allows me to release Homebridge deCONZ and Hue2 without full support for all devices.
Make sure to run Homebridge deCONZ and/or Homebridge Hue2 on a separate HAP server than Homebridge Hue. I strongly recommend to run the new plugins in their own child bridge. I would pair it to a new test home at first, to familiarise yourself with it, and to check which accessories you feel are production ready. Then un-expose all accessories, remove the child bridge from the test home, and pair it to your production home.
In the production home, select an accessory to expose. In HomeKit, you'll now see two versions of the same accessory, allowing you to add the new accessory to the same room, groups, scenes, and automations, while still being able to see how the old accessory is configured. Homebridge deCONZ now creates a migration resource link, containing all the resources it exposes. Homebridge Hue2 will do the same. Homebridge Hue can be configured to take into account these resource links (through the new homebridgeDeconz
and homebridgeHue2
config.json keys). When you're satisfied with the config for the new accessory, restart Homebridge Hue, and the old accessory is removed.
Note that Eve stores the history under the accessory UUID, so the old and new accessories share the history, causing double entries. For accessories with Eve history, you want to make sure to load the history from the old accessory, before exposing the new one, and to remove the old accessory shortly after exposing the new accessory.
Oh, that sounds really horrible.
I guess its safe to just use the old homebridge-hue plugin forever? Re-doing everyhing in the Home App will be a real pain
Yes, that is horrible. That's why I had been delaying the refactor of Homebridge Hue for so long, up to the point where it's no longer sustainable to maintain it.
I cannot prevent that pain, unfortunately, but I won't force you when to take it, nor to take all of it at once. I will stop adding new devices and features in Homebridge Hue pretty soon, after both Homebridge deCONZ and Homebridge Hue2 are ready for mainstream production. I will eventually release a version 1.0 of Homebridge Hue that no longer supports the deCONZ gateway nor the gen-2 Hue bridge, but not within the next six months. Of course, you can continue to run (and even re-install) the 0.x version of Homebridge Hue.
Does using an app like "Controller for HomeKit", with the ability to backup a HomeKit configuration and restore from that backup, address the migration difficulties? I've found that when migrating between Homebridge Hue and the HomeKit integration in HomeAssistant, the backup does not work perfectly due to differences in how accessories are exposed. But perhaps it would be smoother in migrating from Homebridge Hue to HomeBridge deCONZ?
I really don’t know.
@ebaauw can you provide a "migration guide"?
And do you think https://controllerforhomekit.com/features/backup-restore/ can help doing the migration? https://controllerforhomekit.com/2021/06/22/how-to-replace-an-accessory/
As I said before: I really don’t know. I have no experience with Controller for HomeKit.
I’m currently running Homebridge deCONZ in a test home. When I think it’ll be ready to replace Homebridge Hue in my real home, I’ll release v1.0 including a WiKi on migration.
So much appreciate your work and commitment!! It is the most important plugin on my homebridge for about 30 devices. Good luck for the refactoring and thanks for your dedication!
Controller4HomeKit would handle this transition perfectly. I moved all my accessories from a Hue Bridge to deCONZ a year or so ago... after doing so, I matched things up in Controller, and it rebuilt all room assignments, scenes and automations in seconds. It's quite amazing, and imo, removes any worry or concern about migrating between homebridge-hue and homebridge-deconz, etc.
I just installed homebridge-deconz, but there is no configuration at all. I can type something directly in the config editor, but it’s not supposed to be like this I suppose. I disabled the hue plug-in for testing. Anyway, it does not find anything.
I'm just getting into the topic of homebridge and Hue and would like to use the new v2 version directly for this. The v2 must currently be configured in the config file. However, there is currently no wiki or example for this as it exists in the first version in github. Would it be possible to provide a basic configuration for v2 as an example ? I tried a v1 config but that doesn't work. Many thanks Christian
Homebridge Hue2 doesn't yet do anything.
Hi Eric, does homebridge-deconz do anything? I see a stateless switch under devices and in homekit. When restarting homebridge there is no pairing request. The deconz is in pairing mode, but nothing happens. Many thanks so far for homebridge-hue. Regards Peter.
Yes, it’s still under development, but functional. You need to select what to expose in Eve or another HomeKit app, see the README.
Well I'm using the hue emulation on a Tasmota device. :-(
Just saw this in the logs. Thanks for letting us know. Refactors don't get rewarded the way they should. They are better for everyone in the long run, but they often look like an inconvenience to the user. Props for doing it.
Eric, If you need any help on testen homebridge-hue2, please drop me a note! I’m happy to help, really appreciate that you take the time to refactor. This is the only way to keep legacy out of the Homebridge domain.
Wow. That's a lot of work you have to do. So much code and I'm only using it for two devices. And I don't have enough programming skills to contribute anything essential. If you're ever in Frankfurt am Main, your dinner's on me. But maybe you have some better ideas how we can support you.
Same for me in Berlin
Hello Erik,
thank you very much for you developement! I have tested it in my szenario with the Homebridge deConz plugin. After i figured it out how to expose everything in "eve", it worked fine. But in my Homebridge Log im getting every minute this entry. Can you tell me why it getting throwed all the time ?
Thank you!
@TecTolik see https://github.com/ebaauw/homebridge-deconz/issues/30
Does using an app like "Controller for HomeKit", with the ability to backup a HomeKit configuration and restore from that backup, address the migration difficulties? I've found that when migrating between Homebridge Hue and the HomeKit integration in HomeAssistant, the backup does not work perfectly due to differences in how accessories are exposed. But perhaps it would be smoother in migrating from Homebridge Hue to HomeBridge deCONZ?
I migrated from Homebridge Hue to Homebridge Deconz today, and used Controller to create a backup before the migration and restore from the backup after migration. It was pretty painless and the backup restore worked out pretty well (thankfully, since I have 167 accessories). The only accessories that did not restore correctly were my Aqara wall switches, but deleting the service from the backup and reconfiguring those wasn't a big deal. The only thing that did not restore from backup were the custom icons for all of the accessories, but again, not a big deal. I'll post again if I notice any other issues arising from the migration. Thank you for all of your work on this, Erik!
@Ar-eh Any chance you can do a short write up on how to used Controller to do the migration? in particular the sequence of events for the migration. Controller is an expensive app. Likely this migration alone will justify the cost if it's as easy as it seems to have gone for you. I use homebridge Hue to expose of of my Hue Hub connected devices to homekit and also use it to expose my Deconz devices to homekit.
I manage a lot of Deconz devices via homebridge-hue and don't want my life to suck when upgrading to homebridge-deconz. Would this work?
1) Block access to port 51692 on the homebridge machine via a firewall (mine listens on that port for Homekit communication). I think this would make Homekit show "No response" for all homebridge devices
2) In homebridge: Remove homebridge-hue, then install homebridge-deconz and make sure all devices are available
3) Unblock access to port 51692 on the homebridge machine
I'm not sure what you're trying to accomplish with that. The accessories as exposed by Homebridge deCONZ would probably still get new aid
s, causing Homebridge to see them as new accessories.
Hello Erik, is there a timeline when the good old hue plugin stop supporting deconz? I want to be prepared for a migration because when the homebridge stop working my wife is not excited ;-) And in this future, i will end up with two plugins? one for the hue and one for the deconz? and they can co-exists on the same homebridge?
The plugin won’t just stop working, but I’ll stop adding support for new devices and features, and eventually I might publish a version that no longer supports deCONZ.
Yes, there will be two (new) plugins, one for deCONZ and one for the 2nd-gen Hue bridge (using the Hue API v2). They will co-exist in the same Homebridge instance, but I would recommend to configure child bridges for each plugin anyways.
I’m sorry, no timeline. I’m doing this in my spare time. I want to make sure Homebridge deCONZ provides all Homebridge Hue features, before deprecating deCONZ support in Homebridge Hue.
Hi there - I am little bit confused. Should I use the new version of you hue plugin or is this plugin in beta state and the "old plugin" the stable version at the moment?
Hi @ebaauw, I am doing a fresh setup of homebridge with hue. The sole intention for using the homebridge-hue plugin is for exposing the tap-dial switch "long-press" events to homekit. I am not sure if I should start using the new (homebridge-hue2)[https://github.com/ebaauw/homebridge-hue2] plugin for this or I still need to use this old plugin?
Also thank you for your efforts! I went through this thread and the plan look promising.
Homebridge Hue2 is still a placeholder at the moment, so for the Hue bridge, use Homebridge Hue.
Hello .. got this error on hoobs. ...
12.7.2023, 20:26:41HOOBSHueWARNUNG192.168.178.47: request 1: connect ECONNREFUSED 192.168.178.47:1881 12.7.2023, 20:26:41HOOBSHueFEHLER192.168.178.47:1881: not found - retrying in 30s
What could it be ??
Hello .. got this error on hoobs. ...
12.7.2023, 20:26:41HOOBSHueWARNUNG192.168.178.47: request 1: connect ECONNREFUSED 192.168.178.47:1881 12.7.2023, 20:26:41HOOBSHueFEHLER192.168.178.47:1881: not found - retrying in 30s
What could it be ??
you're hijacking a thread, but I have the same Issue out of nowhere (I guess it was an update/upgrade which I've done lately?!) and don't know the solution to fix the root cause. I checked with "systemctl status deconz" all running and on the right port. I could temporarily solve it with "sudo systemctl restart deconz" - ... until the next restart of homebridge, same issue. Maybe this helsp you as well for now....
I've been using Hue v1 and am now looking to integrate v2 to future-proof and test its capabilities.
I'm not sure how to add a child bridge manually or find documentation as per this message: "This plugin must be configured manually using the Homebridge UI Config Editor. Please consult the plugin documentation for instructions on how to correctly configure this plugin."
I must say, I'm quite pleased with the Homebridge plugin so far. Thousands of users, including myself, owe you a big thank you for your work.
Homebridge Hue2 is still just a placeholder at the moment, so for the Hue bridge, use Homebridge Hue.
I'm not sure how to add a child bridge manually
See the Homebridge WikI: https://github.com/homebridge/homebridge/wiki/Child-Bridges.
or find documentation as per this message: "This plugin must be configured manually using the Homebridge UI Config Editor. Please consult the plugin documentation for instructions on how to correctly configure this plugin."
Not sure where that messages comes from, but it's probably the result of Homebridge Hue2 not yet containing a config.schema.json
, which defines the Settings form to the Homebridge UI.
Homebridge Hue2 is still just a placeholder at the moment, so for the Hue bridge, use Homebridge Hue.
Ok, I'll just let it stay under my plugins as disabled then.
Not sure where that messages comes from, but it's probably the result of Homebridge Hue2 not yet containing a
config.schema.json
, which defines the Settings form to the Homebridge UI.
It happens whenever I click settings or bridge settings in Homebridge plugins.
Thank you for the fast reply
UPDATE AUG 2023 I've bumped Homebridge deCONZ to v1, as I think it's production ready. I concluded my migration to Homebridge deCONZ in May, and have been running Homebridge deCONZ exclusively for three months now, without any issues.
Consequently, Homebridge Hue support for deCONZ gateways has been deprecated, meaning you should start planning a migration to Homebridge deCONZ. For now, Homebridge Hue continues to expose devices connected to a deCONZ gateway, but I won't be adding support for new devices nor implement any feature requests not supported by the Hue bridge.
Note that Homebridge Hue2 is still just a placeholder, so continue to use Homebridge Hue to expose devices connected to a Hue bridge.
Consequently, Homebridge Hue support for deCONZ gateways has been deprecated, meaning you should start planning a migration to Homebridge deCONZ.
Can you write a guide how to do the migration?
ui
command-line tool, un-exposing and exposing devices, changing log level, how to expose devices, etc.homebridgeDeconz
to the ID of the migration resourcelink by Homebridge deCONZ from step 8.hosts
, remove the IP address and port of the deCONZ gateway from the array. When using automatic discovery, set noDeconz
.Thank you @ebaauw for the very detailed migration guide! 🙏🏻
I‘m a little bit confused about step 7
Un-expose all devices in Homebridge deCONZ
How can I do this? Also when I add the gateway it automatically expose all devices right? How can I prevent this to only expose one by one as you described starting from step 9?
What does this „migration resource link“ thing? I do not completely understand it. I mean I could also just add the new gateway, remove the old one from HomeKit and resetup everything?
How does the „migration resource link“ help here? And also when does it show up and where?
Thank you already for your help!
I‘m a little bit confused about step 7
Un-expose all devices in Homebridge deCONZ How can I do this?
Issue ui put /accessories/
id'{"expose": false}'
for each accessory. Hence: familiarise yourself with Homebridge deCONZ, notably step 5.
What does this „migration resource link“ thing?
It lists the resources exposed by Homebridge deCONZ, so Homebridge Hue can stop exposing these.
I mean I could also just add the new gateway, remove the old one from HomeKit and resetup everything?
Yes, you could, if you want to do a big-bang migration. Depending on the size of your HomeKit home, this might take you a full day or longer.
How does the „migration resource link“ help here? And also when does it show up and where?
It's a /resourcelinks
resource on the deCONZ gateway, maintained by Homebridge deCONZ, and read by Homebridge Hue. Homebridge deCONZ writes a log messsage, whenever the exposed devices change, or when issuing Identify on the gateway accessory.
[8/12/2023, 11:30:07 AM] [deCONZ] pi5: migration: /resourcelinks/3: 35 resources
You would enter the ID, 3
, in the Homebridge Hue configuration.
Works like a charm! Thanks. I made a fresh Installation btw. The HomeKit wizard is really simple. It gave me for all devices a popup to already move the new fetched devices to existing groups.
I was wondering if there is a plan to bring back the adaptive live adjustments to the new version. For Homebridge hue there is a option to make the lighting not too red.
Also there is an option to set up what type of devices should be exposed. As far as I understood with the new version it only allows me to not expose devices that I set to expose false one by one and not like „all switches“?
I also recognised that when I turned on adaptive light for a group in a room like (lamp 1-3 grouped into a single lamp group) the brightness doesn’t set correctly (when I set the group to brightness 50 is lacks means doesn’t set up to 50 directly. just after a few tries). For single lamps this works for adaptive light. When adaptive light is off it works also for groups. I was wondering if this is more a Deconz problem itself. I mainly use ikea lamps btw.
Please stay on topic and don't pollute this issue with other questions.
What will happen with the automations from the exposed devices? Do we need to "link" them again to the correct device/automation?
HomeKit sees the devices exposed by Homebridge deCONZ as different accessories as when exposed by Homebridge Hue. When you no longer expose the accessories by Homebridge Hue, any associations with HomeKit rooms, groups, scenes, and automation are lost. You need to change these to include the Homebridge deCONZ accessories instead.
I‘m a little bit confused about step 14
i somehow cannot delete the "old" deconz Gateway. It is always required by the Hue plugin to press the link button. The ResourceLink is inserted correctly.
How do I remove the Deconz from the Hue Plugin configuration?
That’s a good point: as long as you use auto discovery, Homebridge Hue will discover the deCONZ gateway and block Homebridge startup, until it has connected to it. You can disable auto discovery by listing your Hue bridge(s) in config.json, under hosts
, but that only works when the Hue bridges have a fixed IP address.
I probably need to cut a Homebridge Hue release that (optionally?) only auto discovers Hue bridges.
UPDATE DEC 2023
UPDATE AUG 2023
Homebridge Hue is my oldest plugin. I started working on it in February 2016, and published the first version on GitHub in October 2016. I added support for deCONZ in the summer of 2017, including web socket notifications. In February 2020, I added HTTPS support for the gen-2 Hue bridge, followed by API v2 push notifications in December 2021. Over that period, the number of different devices supported by Homebridge Hue has increased dramatically, fuelled by the consumerisation of Zigbee devices. Starting with (then) Philips Hue, OSRAM Lightify, and IKEA Trådfri, followed by Xiaomi, and many OEM devices, and even devices from Lidl and Aldi. The code base has become increasingly complex over the years, and is due for a refactor. Homebridge Hue still uses Homebridge's traditional static platform accessory model (see #4), where most recent plugins use the dynamic platform accessory model, which offers many benefits. The Hue and deCONZ APIs have been diverging, and their v2 APIs are completely different, making it increasingly hard to combine Hue bridge and deCONZ gateway support in a single plugin.
Homebridge Hue is also my most used plugin. I don't collect any statistics, but the NPM registry mentions up to 8,000 weekly downloads, and GitHub Insights lists over 3,400 unique visitors. Homebridge Hue is also, by far, my largest plugin; I'm exposing over 140 accessories from my deCONZ gateway. I would take me half a day to reconfigure HomeKit if those accessories would be removed from HomeKit and re-exposed. Because of this, I've been hesitant to refactor Homebridge Hue. While I can technically manage to expose my accessories using a dynamic platform plugin, without HomeKit losing them, this is tricky and requires a deep understanding of the internal working of Homebridge and HAP-NodeJS. For me, a big-bang update would be out of the question, as I think for most users.
Going forward, I will create a new plugin for deCONZ, homebridge-deconz, and a new plugin for Hue APIv2, homebridge-hue2, while keeping the current Homebridge Hue plugin. These three plugins can be run in parallel, using different child bridges, connecting to the same Hue bridge(s) and/or deCONZ gateway(s). This allows you to setup a new HomeKit home for testing. By connecting the child bridges for all plugins to the same Home, white- and blacklisting can be used to move accessories over to the new plugins one-by-one. This will also allow me to release work-in-progress versions of the new plugins, not yet supporting all device types. Eventually the current Homebridge Hue plugin will drop support for the gen-2 Hue bridge and the deCONZ gateway. After that, I might release a new version with the dynamic platform accessory model. I don't think there are enough users left using the gen-1 Hue bridge or HA-Bridge or Tasmote to justify the effort needed to provide a gradual migration path.