homebridge 0.5

[KhaosT]

It has been a while since the major changes happened with homebridge. I bet Nick didn't think homebridge community would grow this fast when he first started it (I certainly didn't 😆).

Going through existing issues, it looks like people are having a lot problems with initial setup, unexpected plugin crashes and HomeKit configuration disappear (#874). To address those problems and improve the overall experience, I'm proposing the following changes for the next major release of homebridge. This is a very rough draft of the plan, feel free to comment and add new things to the list. I'm relatively free over the next few months so hopefully I'll be able to implement most of these.

• Plugin OOP
  • Most of existing plugins will not be affected since API should remains the same
  • Automatically reload when plugin crashes
• Mandatory accessory caching
  • Optionally opt out?
• Web Interface
  • Manage cached accessories
    • Remove them
    • Control them?
  • Manage plugins
    • Config plugins
      • Interactive based on existing configuration service?
      • Field based?
    • Install plugins?
• Dynamically splitting accessories across multiple bridge instances
• Plugin certification
  • May just be "recommended plugin" in the plugin catalog instead of mandatory requirement for homebridge to load the plugin
  • Code review for plugins
• New plugin types
  • Observer (Read only)
    • Have homebridge poll endpoints with interval or go with notification based solution?
    • Data logger
  • Manager (Read write)
    • Automation
    • Plugin management

---

Nice to have

• ES6 rewrite
• Mac Wrapper
  • A macOS app that can run directly with bundled nodejs env
• Raspberry Pi setup script
  • For Raspbian

---

PS: Anyone interested in writing a proper web interface?

[gismo141]

@KhaosT:

The Webinterface: see my plugin homebridge-server which supports only bare-essentials at the moment but it works for a lot of different setups already.

The Raspberry Pi setup script: see homebridge-raspbian-installer for the installation and basic setup of a running homebridge-environment.

At the moment I'm trying to implement additional functionality and flexibility.

[KhaosT]

@gismo141 Yeah, I saw homebridge-server before :) It's really nice you're doing that. I didn't know homebridge-raspbian-installer exists before, guess that's a done for the last task.

[snowdd1]

Good ideas altogether! May I add a filter to avoid illegal response values? It is the next most identified root cause for homebridge being banned from HomeKit as far as I could see.

[KhaosT]

@snowdd1 that's planned in HAP-NodeJS level ;)

[ehusar]

As a new homebridge user in just the last 2 weeks I can comment on what has been the most frustrating parts for me personally.

1. The install of Homebridge is a bit hit and miss. At least early on that is the case. Its not that the instructions are off but more missing a step here or there and then it all fails. I can say that doing it about 5 times now I think I can do it without issue the first time. It is more about getting everything required installed before you do the Homebridge install. The instructions today have you jump out to the README and then back to the instructions to finish up. Getting the "official" install streamlined would help new users. Once document to rule then all. An RPi script would be really nice.

2. plugins. Getting the syntax right is key. I can't tell you how many times I have editted the config file since I missed something silly. My config is also getting really big now. Automating edits to this through a GUI would be awesome.

Those two are really the big ones for me and it is not necessary the issue of Homebridge at all.

A Datalogger woud be awesome. I'm currently looking to log all room temps to a cloud solution since I can't do this in Homebridged.

"Dynamically splitting accessories across multiple bridge instances." This is also something I think would be a big plus.

[cocacola89]

Hi there, @KhaosT!

As for me:

Web Interface I would really upvote for an Accessory interface, which would include basic control functions like the ones from official Apple or Matthias Hochgatterer's Home Apps as well as administrator-ish functions. From what I've learned, some people are switching to Home Assistant + homebridge-homeassistant setup to have a nice interface on non-iOS devices.

As for Configuration interface: from my experience this can be really tricky. You can spend a lot of time and effort on developing such UI, but people would still go with the config file option. As the single configuration file grows it becomes really clumsy to edit it, but it can get a lot more clumsier within the UI (field-based, composite editor-based, etc.)

Automatically reload when plugin crashes I don't really think that is good idea. When plugin crashes, it may leave a whole lot of junk in memory and its internal state, so it may not work properly after reloading anyway. I'm using Windows setup for Homebridge with node-windows that restarts Homebridge after crash and auto-reloads it once a day. I believe a lot of people are using quite the same setup depending on the platform. I think that reloading the whole Node.js app is a better solution, and it should be left to OS-level, not application level. With Windows setup you can customize restart behavior via standard services.msc interface (e.g. fist, second, subsequent failures), which is quite nice.

I think though, it would be a good idea for Homebridge to disable buggy plugins after a number of subsequent crashes - that's something OS-level managers wouldn't be able to achieve.

Mac Wrapper I would personally go for a more cross-platform solution here. There are plenty of frameworks that include bundled Node.js with native OS UI integration, like NW.js, Electron, etc.

New API I would really love to see new API in Homebridge:

• Programmatic:
  • Access other plugin/platform instances and receive events (e.g. Plugin loaded, Initialized, etc.)
  • Access accessories from other plugins and execute actions (e.g. turn lightbulb on/off, etc.)
• REST
  • Access accessories from other plugins and execute actions (e.g. turn lightbulb on/off, etc.) by sending a relatively simple HTTP requests.

I've encountered a desperate need for those API trying to bind Aqara Switches with Philips HUE bulbs. Aqara API uses a reserved port number, so it allows only one instance of application machine-wise. As I'm using homebridge-aqara, it was either my separate automation script or homebridge-aqara on one machine, so I decided to hook to homebridge-aqara platform instance from my own Homebridge plugin. I've discovered though, that Homebridge exposes API for getting constructors for platforms via platform() method, but not to platform instances, so I had to do some prototype-magic to get to Aqara platform instance. After I'd done this, I wanted to send commands to Philips HUE lights, which were controlled by homebridge-philipshue. Another bummer there - I couldn't find a way to get an accessory registered by other plugin, so I had to go with using Philips HUE API directly (that means separate configuration, registration on HUE bridge and so on).

This case may not be a common one, but I think it would be nice to have such API to enable richer integration between plugins. As for REST API - just a nice option to control your accessories via simple scripts without mDNS of any kind 😄

Thanks a lot for all the hard work @KhaosT, we really appreciate it!

[npalamar]

Hi, @KhaosT :)

Web Interface: Will be cool if it will be console daemon for managing homebridge plugins + web for this daemon or even this scheme: console daemont > rest api > rest client :) Will be cool!

[snowdd1]

May I add a filter to avoid illegal response values? It is the next most identified root cause for homebridge being banned from HomeKit as far as I could see.

@snowdd1 that's planned in HAP-NodeJS level ;)

If done something with very crude code to get the enumerations from the characteristics (prototype), you might have a look at it: https://github.com/snowdd1/homebridge-knx/blob/master/lib/knxaccess.js#L372

It would be very good to use the characteristic extractor/generator to get a member with the valid values in a member that has the same name for all characteristics, instead of giving them the name of the characteristic (see https://github.com/KhaosT/HAP-NodeJS/issues/141#issuecomment-146312250 ) And the best if HAP-NodeJS will take care of it instead of all the plugins have to do it.

[MrsDelish]

Would a recommended plugin wiki page, with a list of each type, maybe be a good idea

A barebone sample/skeleton plugin with recommended error handling and such, could help plugin quality.

@gismo141 https://github.com/macnow/homebridge-raspbian-installer looks really nice, it should be in the readme.

The steepest learning part of my first install was probably plugins with typos/syntax errors in the sample config.json, after finding http://jsonlint.com/ that went better. well that and just bad luck, I followed the wiki and went for Node.js LTS it was version 6.2.1 att iirc and homebridge would not start, just kept crashing, upgrade to latest stable 7.x many hours later and everything was working.

[RienduPre]

Any idea when the new functionality will be available ?

[KhaosT]

I think right now please expect sometime between March and April for this to land.

[JangoBritt]

Also... better stability would be nice. Keep having to remove and re-add my homebridge because... something?

[VeniceNerd]

I'm so happy Homebridge development is active. I love it! I was finally able to integrate all of my Insteon devices via the ISY994 and the excellent ISY-JS Homebridge plugin.

One suggestion I would have it to make it easy to change accessory device types. Let me configure what HomeKit device type should be assigned to select accessories. The default isn't always what makes the most sense.

For example: I have a towel heater connected to an Insteon DIN On/Off module. It is imported as a light. In most cases that would be correct, but in this specific case I'd like to set it to "heater" or at least "outlet" if there is no heater device type in HomeKit.

A more user friendly interface would be much appreciated as well, of course. ;)

[snowdd1]

Hi @venicenerd
Your are mixing plugin and homebridge functionality.
The type of accessory (in fact, of the service) is defined in the plugin, and has to be, because meaningful characteristics depend on that (except for a few exceptions that use the same characteristics)

[NorthernMan54]

What you are discussing is an existing feature of the Home app. If the plugin define's the service as outlet, you can change the type in the Home app between outlet, fan and light. You could ask the plugin developer to make this change.

Sent from my iPad

On Jan 28, 2017, at 3:45 AM, Raoul notifications@github.com wrote:

Hi @venicenerd Your are mixing plugin and homebridge functionality. The type of accessory (in fact, of the service) is defined in the plugin, and has to be, because meaningful characteristics depend on that (except for a few exceptions that use the same characteristics)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[snowdd1]

Right, Outlet service is quite flexible. Lightbulb is not. That change is up to the plugin developer.

[jojost1]

Is it possible to create a HomeKit-hub alternative like the AppleTV4 or iPad? Probably not, but asking for 0.5 ;-)

[KhaosT]

This is not possible due to technical limitations.

Sent from my iPhone

On Jan 31, 2017, at 3:21 AM, Joost van den Akker notifications@github.com wrote:

Is it possible to create a HomeKit-hub alternative like the AppleTV4 or iPad? Probably not, but asking for 0.5 ;-)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[pdlove]

I like the ideas I see here. I really want to see the Manager-style plugin because that allows a good web control or an advanced rules engine to be created. On the Observer, I'd like for the plugin to be able to subscribe to notifications... so that if a Characteristic changes, it sends the change update to the plugin.

I don't suppose it's possible to assign the room in HomeKit from homebridge?

[JangoBritt]

Gawd, wouldn't that be nice! Lol. Nothing quite as annoying as having to re-define 50 SmartThings devices every time HomeBridge need to be re-added to HomeKit. Lol

[JangoBritt]

Also, I wonder if it would be possible to add devices individually, rather than adding the HomeBridge and having to replay the config on each, if you only actually need half of what is being pulled across by HomeBridge (i.e. Temperature but not Humidity, Door Lock but not Tamper Sensor)

[pdlove]

@JangoBritt The discussion on storing the room assignment in HomeKit always ends the same... "Feel free to build the app on the iPhone that does it because we can't push it over the HomeKit protocol."

That does bring up a good point though... It would be nice to suppress the Services being presented to IOS by type but still pass all of them to the datalogger plugin that someone will eventually create.

[KhaosT]

@pdlove @JangoBritt can you guys describe how frequently do you need to remove the homebridge (accessory) from HomeKit and redo the room assignment? And if possible, can you tell why you have to do that frequently?

If the situation is really terrible and there is no other way to address this other than offer an option to restore space assignments from other source, I can consider add this to the planning and add the restore part in Hesperus. (Not promising anything yet 😅)

[pdlove]

As a plugin developer, I have to far too often. I think giving the ability to remove defunct accessories and adding better trapping to HAP-nodes that will prevent invalid accessories from purging the whole list in HomeKit will prevent the average user from ever having to redo it.

[JangoBritt]

Possibly the ability to add each platform separately into HomeKit, rather than the whole bridge, might also help to diagnose if one of the plugins is causing it

[snowdd1]

@pdlove
Don't you have a separated instance for developing? I have two raspberries, one productive which only needs tampering with for configuration or OS updates, and a development instance that hardly is on a whole day - and I use an old iPhone 5 with a separated iCloud account for developing, so I do not have to connect my primary iCloud account to dev ever. That way it is totally stable, usually production runs for several months.

[ebaauw]

Just stumbled upon this thread. If I may offer my suggestions:

Dynamically splitting accessories across multiple bridge instances

Yes! Assuming this will workaround the 99 bridged accessories limit. I currently check that my plugin doesn't cross this limit, but I cannot take into account the accessories exposed by other plugins.

Plugin certification May I add a filter to avoid illegal response values? It is the next most identified root cause for homebridge being banned from HomeKit as far as I could see.

Yes. In general, any checks / assurances that would keep HomeKit from choking on homebridge. Not just out-of-bound values, but also limits like >99 bridged accessories, >99 services per accessory. A check on the utilisation of the 1MB CKRecord limit would be nice, but might be challenging (see https://forums.developer.apple.com/thread/64682).

Access other plugin/platform instances and receive events (e.g. Plugin loaded, Initialized, etc.) Access accessories from other plugins and execute actions (e.g. turn lightbulb on/off, etc.)

That doesn't feel right to me - plugins should work independently, except for:

New plugin types Observer (Read only) [...] Manager (Read write) [...]

I particularly like the idea of automation. Of course, we wouldn't be able to access and execute HomeKit triggers and replace the HomeKit Hub, but providing (a more stable) alternative sounds interesting.

Mac Wrapper - A macOS app that can run directly with bundled nodejs env Raspberry Pi setup script

What about providing a container including nodejs, homebridge, (certified!) plugins, and dependencies, incl. automated startup/restart of homebridge? They're running homebridge in a Docker container on Synology, see: https://github.com/marcoraddatz/homebridge-docker/blob/master/readme.md.

[ebaauw]

Another thought I've been entertaining is the following:

During the development of several homebridge plug-ins, I found myself copying a significant amount of code between these plug-ins. I'm tempted to move this generic code into a separate library package, in the hope that it will result in easier development and maintenance of actual homebridge plugins, as these only need to provide the device-specific code.

This generic code would include logic for:

• Initialising the plug-in, including setting up the links to homebridge and to HAP-NodeJS;
• Restoring homebridge v2.0 dynamic platform accessories from persistent storage;
• Setting up a Universal Plug and Play (UPnP) listener for device discovery;
• Setting up a heartbeat for device polling;
• Setting up callback functions and event handlers for interaction with homekit;
• Error handling and logging.

I'd like to use a modern programming style for this, with the following aspects:

• Using a Model/View/Controller-like design pattern, where the homebridge plug-in serves as Controller, and the devices as well as the homekit database serve as Models. Obviously, the Views are provided by homekit apps, each of which has their own Controller on the same Model, the homekit database;
• Using classes instead of prototypes. The generic code is provided by (abstract) superclasses, implementing the Controller logic to the Model provided by the homekit database. Each of these classes acts as a delegate for their Model equivalents, provided by homebridge (Platform, PlatformAccessory) or HAP-NodeJS (Service, Characteristic). An actual plug-in, based on the library, would create subclasses of these, implementing the Controller logic to the Model provided by the device;
• Using promises instead of Node.js callback functions. The Pyramid of Doom rears its ugly head as soon as you try anything more sophisticated than a simple pass-through plug-in.

Instead of doing this in a separate library, couldn't, shouldn't this be part of a new homebridge api?

[JangoBritt]

@KhaosT

@pdlove @JangoBritt can you guys describe how frequently do you need to remove the homebridge (accessory) from HomeKit and redo the room assignment? And if possible, can you tell why you have to do that frequently?

So, judging by when I posted that... 17 days is my last gap.

I'm not doing anything differently, nothing in particular is showing up in the logs. No devices added, no devices removed, it just... disappears.

I've tried rebooting the phone, tried rebooting the Mac, Once it gets into this state, it seems to stay there until I have to re-load the homebridge :-(

I wish I knew where else I could check to see what the phone is trying to do when it looks for HomeBridge!

It's got to be the homebridge server somehow... because it's finding my Philips Hue bridge just fine

[ebaauw]

On the room assignment: rather than developing a HomeKit iOS app, I've been playing with the idea to expose all the Lightbulb services for the lights in a room in one "room" accessory, rather than in a separate accessory per light, see https://github.com/ebaauw/homebridge-hue/issues/43. My main objective is to workaround the 99 bridged accessories limit, but as a bonus, you'd only have to (re)assign one accessory per room.

The HomeKit apps I know show an "accessory" per service, so from a user perspective you'd hardly see any difference. As far as I can see, the only functionality you'd lose is the ability to Identify an individual light, because it will no longer have a separate AccessoryInformation service.

The Hue bridge persists which lights belong to which room in special Room groups. We could even take this further and use Hue bridge resourcelinks to persist which motions sensors, taps, dimmer switches, clip sensors belong to which rooms. In that case, you'd also lose the BatteryService service per device. I could add the BatteryLevel and StatusLowBattery to the device's (main) service, but I'm not sure if Home will understand that. On iOS 10.3 beta 2, Home shows the BatteryLevel under STATUS for programmable switches and sensors when defined as a separate service.

[JangoBritt]

The Hue app works by sync'ing the rooms, scenes, and lights. I guess a Homebridge iOS app could do something similar, but it would have to be an actual app, and not a webApp, meaning that it would have to be presented via the App Store. It will never get approval, because of the way HomeBridge is flaunting the HomeKit encryption rules.

[ebaauw]

It wouldn't have to present itself as an app specifically for homebridge.

When syncing rooms, the Philips Hue app syncs the room assignment just as happily for the (non-Philips) lights exposed through homebridge-hue as for the (Philips-only) lights exposed through the Hue bridge. (See https://github.com/ebaauw/homebridge-hue/issues/55, if you want to know how). Neither Apple nor Philips intended this functionality, let alone approved this.

[JangoBritt]

Here's an odd thing... is happened again (twice) since my last post. However, I've noticed this time that the iPad (home hub) is still connecting just fine!

How can I reset the iPhone, without losing everything?

[Glavin001]

Any progress on the Manager (read/write) automation plugin? I would love to use it and be happy to contribute! I am not sure how one would implement a reading of other HomeKit accessories. My goal being able to replace HomeKit Hub with a Manager server evaluating rules and executing actions. Please let me know how I can help! Thank you!

[pdlove]

Is anyone working on the management plugin type yet? I've got a few ideas for how to handle it but I'm still working out how Homebridge itself is put together. If someone more knowledgeable than me is working on the plugin framework then I want to back off and just be patient. :D

[KhaosT]

Right now the plan is to expose a new API for management (and observation) plugin to query all accessories from homebridge. The return result will be an array of Accessory object. Inside you have the same hierarchical structure (Accessory -> Service -> Characteristic). On Characteristic object, there will have a few new methods (readValue, writeValue) to handle the management role. Notification (aka value update) will be emitted as event on Characteristic object so plugin can observe value updates.

Does this design fulfill your needs for management plugin?

[pdlove]

It fully covers phase 1 except that a management plugin should be able to access the platform's configurationRequestHandler which is something that can be done through the current API but also send the updated configuration back for writing to config.json. This would enable a complete management Website to have full configuration/management of Homebridge. The only think that would be missing would be a way to unload/reload a specific plugin on the fly.

I think what I consider phase 2 is more of a specialized request as it changes the connection between the accessories and the bridge extensively and really is more of a core functionality change rather than an API. I'm going to work up a hacked together proof of concept to really demonstrate what I'm talking about and to become 100% sure of what I'm talking about.

[pdlove]

Here's my thought for how I'd like to see the flow to make this plugin the all powerful program to dominate my home: https://docs.google.com/document/d/1M9YN-gwR4NxGQU1aCZOXOk7Z446Pm_0Wkwha2TuHIwA/edit?usp=sharing I think I almost have enough understanding of how Homebridge itself is put together to be able to test doing all this... but I know it'll take time to get something reliable and secure together. I apologize for the funky Google Docs formatting.

[lcorsini]

I installed homebridge today, and so far it works well (needed to interact with Hue) In the next weeks I should receive my aqara kit and, like @cocacola89 said a way to implement interaction between plugins could be very interesting.

An aqara switch could turn on a hue bulb, or a humidity sensor (exposed from aqara) could trigger turning on a smart plug connected to hue, door sensors could trigger an http action or simply turn off a Netatmo thermostat.

@cocacola89 how did you manage to connect aqara switch to hue bulbs?

[KhaosT]

@pdlove ah I see what you mean, we are thinking differently when it comes to "management plugin". When I say that, I was mostly mean plugin that has the capability of controlling home accessories and states. For the configuration and management stuff for homebridge, I was thinking about having homebridge provide a RESTful API set alone with a web interface built on top of the API. The later part will be the one dealing with interacting with plugin's configuration handler, config management and bridge association.

The idea of value transformer is interesting and I can definitely see some people find it's useful.

I have a question about "accessory has a property of a bridge array that defines the name of the Bridge(s) it is assigned to", is there a particular reason why an accessory/plugin should care about which bridge it belongs to?

[pdlove]

It's not so much that the accessory cares. I was just thinking through saving/loading the cached accessories and which Bridge they belong to and thought it easiest to do it on the accessory so the Bridges could be created and then the Accessories added. The Rest API for config could be used by a management plugin as well so that still does what I'm wanting.

The transformation Bridge has a number of purposes. My original intention was just to combine the lights of my living room in a simple way that isn't HomeKit specific. It could be done simply by making the bridge a plugin with the HAP as the default.

[snowdd1]

Hi Tian, I just wanted the topic to come up in the vast pile of issues here. I am still eager to get my hands on the new API once it's there! Any idea when it might surface? Thanks, Raoul

[KhaosT]

@snowdd1 I apologize for the lack of update... I got occupied by some other stuff and didn't make too much progress on this... I'll try to work on this soon. Sorry for the delay 😅

[ghost]

Any updates/ETA, when this will go online? Can't wait to implement it, into my HomeKit environment!

[rcarmo]

Just to add another voice to the crowd asking for an API to set accessory state (I have a lot of stuff I control via IR but can only determine whether it's on or not by polling through various mechanisms).

Also, a nice, clean way to disable accessories in the config file would be great (enabled: false, etc.)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity, and will be closed if no further activity occurs. If this issue was overlooked, forgotten, or should remain open for any other reason, please reply here to call attention to it and remove the stale status. Thank you for your contributions.