search

home-assistant / architecture

Repo to discuss Home Assistant architecture
310 stars 102 forks source link

Linking two or more Home Assistant instances #246

Closed danielsjf closed 1 year ago

danielsjf commented 5 years ago

Context

Home Assistant is great in grouping and controlling local and webbased devices/services. However, there are cases in which local actually means two or more unconnected networks. A few examples:

In all these cases, it would be great if two different versions of Home Assistant could talk to each other. Since one will always be the aggregator of the data, I will call this main instance of Home Assistant the master. The second device will be the client.

In the past there have been (and still are) a few attempts to make this work:

Both the first and the third option currently work, but both aren't hassle free setups and aren't integrated in HA. Therefore they might break if HA is updated.

Decision

The proposal is mainly a retake of the comment of @balloob.

In short, the master and client would communicate via the websocket API. The client would push state updates to the master when they change.

The MVP would only relay sensor data from the client to the master. It would require the following changes:

  1. Extend the websockets API with new functionality. It should be able to create entities, update their values and remove them again in case the connection stops.
  2. Create a new component: outpost that is set up on the client. It will duplicate all client entities on the master via the websocket API and push all state updates on the client to the same master instance.

Further developments could add the following functionality:

Consequences

Potentially more maintenance of the websockets API but this is already used heavily by other components.

Swamp-Ig commented 5 years ago

The MQTT event stream almost does this. It sends the state changed events from one HA instance to another and allows these to display on the front-end.

What it doesn't do is replicate the states themselves, so you can't then query the states. It also doesn't transmit service calls back to modify the states.

I don't think a master-slave arrangement is really ideal however, I think each HA should be responsible for its own states, not transmit them off site. You don't really want you holiday house's watering not to go on because the internet is down, this is the whole reason why people are wanting to use local control rather than "in the cloud".

The underlying technology here isn't actually important, but to me the outcome should be:

  1. Each HA instance manages its own states, and the service calls affecting them
  2. On connection, the state data is transmitted to the remote site. This state data is marked as remote, and has a connection ID associated with it.
  3. State change events are also transmitted to the remote site so the remote site can update its local copy of the state. These would need to be queued for transmission in case the connection is patchy.
  4. The available services for a state are also transmitted to the remote site.
  5. Service calls on the remote site are intercepted and transmitted back to the owner of the state, and affect the local state. The remote state has to wait for the state change data to come back so that its own state is updated.

Really automations should really be hosted on the HA that uses them, so they don't get disconnected when the internet goes down, but if users really want to rely on their internet connection then this method would work.

danielsjf commented 5 years ago

Indeed, I agree that automations that can run locally always should. It is not the goal to replace these.

In my opinion, the goal should only be to aggregate control for the user. Therefore, in order of priority, the steps are

  1. Only viewing the data (or occasionally setting states) in one interface, which is less critical when the connection goes down.
  2. Add automations that cover two instances (e.g. alarm goes of in house if intrusion is detected in pool house). These automations only run on the master instance. If we also want to implement this in a later stage, we should indeed think of a fallback. However, this could be very similar to dropped sensor data on one HA instance.

I don't have a strong opinion on MQQT versus websockets. However, given that steps are made to support the web things API, there could be gains in an implementation with the websocket API (which is also used for web things and might share a few developments).

Swamp-Ig commented 5 years ago

If we have successful service call transmission, then there's no reason why you can't run remote automations, it's more just that it wouldn't be the best idea.

The main issue with not transmitting service calls is that then you'll have a read only state, which we don't currently use much in the front end, so the display is likely to be buggy.

Hedda commented 5 years ago

Also see the somewhat related High-Availability architecture ideas/discussion here about linking instances for redundancy -> https://github.com/home-assistant/architecture/issues/152

That is, could maybe achieve High-Availability if could link two or more Home Assistant instances and also add a fail-over feature.

marcandre83 commented 4 years ago

I don't really understand, is there a possibility to call a service on the remote site? I want to call the remote site's notify-service because the remote site is paired with my iPhone and the HA Cloud.

blokpardi commented 3 years ago

This is a showstopper for me so I really hope someone can figure this out. I'm looking to make the switch from SmartThings but really need to be able to use phone-based presence with two instances. I got my Pi this week and got the first instance setup at one place, but when I went to look at setting up a second instance at another house I realized there's no way to link them. One of the big use cases for my home automation is to be able to switch between the two and see what's going on at the other place, as well as trigger automation based on presence (or lack of it) at each place. SmartThings has a UI affordance to do this and it's pretty seamless. It seems like Home Assistant should be able to do something similar. Just setup two separate instances (URls) in the app, and when you switch it just uses whatever instance you switched to. Then the instance should handle all the presence stuff as it does now.

genebean commented 3 years ago

I'm looking at this now too as I've recently set my house up with HA and have already set a zone for my church. I want to use presence to turn things on and off when I'm in my office there but also want to manage everything via a single UI.

It seems like the web sockets method of Home Assistant Remote should work if both instances are hooked up with HA Cloud or the MQTT method talked about in https://link.medium.com/ExIn8GTHOcb should be doable if combined with a VPN between sites.

Ideally though, HA would support managing multiple instances centrally so that the remote/secondary ones could be more of a bridge than a full HA setup. Central management would also allow for installing updates, integrations, and add-ons in a consistent way across nodes. Following the bridge idea, if the main and supplemental instances are on the same network then the could link up automagically via discovery. If separated they could link via ZeroTier or WireGuard prior to being discovered. In my head, the non-primary instances would function much like a Hue Bridge with the addition of a minimal UI for setting up the link.

JonathanTreffler commented 2 years ago

Any updates ?

BradleyFord commented 2 years ago

I think I have seen this before and it was declined as adding way to much complexity to the code base, and what is the benefit of have integrations and data sharing that cross 2 systems?

Could they not be independent systems, and then you login to each individually. So maybe all that is needed is the web interface and app having the ability for you to switch sites.

thecode commented 2 years ago

I think I have seen this before and it was declined as adding way to much complexity to the code base, and what is the benefit of have integrations and data sharing that cross 2 systems?

Could they not be independent systems, and then you login to each individually. So maybe all that is needed is the web interface and app having the ability for you to switch sites.

Ignoring complexity (agree with you) I think that there is a large benefit of additional HA instances which communicate with devices. reasons for that:

  1. Some devices needs local connection to an hardware such as serial based projectors, AVR devices - I have a serial based AVR device which I ended up in adding an ethernet -> serial adapter and wrote my own code to integrate that into HA, but ideally I would not be needed. PI devices are cheaper than serial ethernet adapters and even having a PI for a single device is still cost effective.
  2. Sometimes it is impossible to have a single network coverage for RF based devices. Bluetooth is a good example were you can't add repeaters, but it may also useful types which does support repeaters, I had a discussion recently with a user with a house spread over multiple floors and a complicated solution to make a single ZigBee network.
  3. It is not simple to switch on the mobile app between accounts and also has many drawbacks. Let's say you want to turn off all lights, you need to do that twice, it is also not possible to make automations which uses the two instances.
ehn commented 2 years ago

Another use case for this would be not having to connect multiple homes to the same cloud service. For example, if you want to track health data from Withings and presence from iCloud, it's redundant and, yes, additional complexity, to have to do that on multiple HA instances.

(In my opinion, it would be great if all cloud connections could go via Nabu Casa and then be made available to all connected HA instances, but I understand more privacy-concerned users might not want that. Also beyond the scope of this issue.)

Jc2k commented 2 years ago

I don't think we need multi HA for the apps to support multiple homes. That's a feature request for the apps.

I don't think it will ever really make sense to bolt on full and total control of one HA from another HA (like managing automations and pairing and stuff). UX wise it's quite a large thing to retrofit, short of a ground up redesign of the UI I think it would just lead to a confusing mess.

I do think one HA showing entities from another HA is interesting and useful. We can already do that to a certain extent with MQTT or HomeKit. I quite often have one HA controlling another HA by using homekit bridge on the "remote" side and homekit_controller on the "main" instance. To extend that between homes you'd just need a site to site VPN or something. It looks like the remote_homeassistant custom component might do something similar but without being limited by external schemas. All of these would let you forward just a subset of entities and events rather than forwarding everything an polluting the parent HA.

blokpardi commented 2 years ago

The SmartThings app does this and it's not a confusing mess. It simply lets me manage two independent instances (one for each house) and as far as the app is concerned there is no communication between the two. The app acts as a hub between the two instances, providing a common place to look at the state of each, manage devices and automations, provide device based presence, and surface alerts and notifications for both instances. The interface stays clean because I can only look at one instance at a time.

thecode commented 2 years ago

The SmartThings app does this and it's not a confusing mess. It simply lets me manage two independent instances (one for each house) and as far as the app is concerned there is no communication between the two. The app acts as a hub between the two instances, providing a common place to look at the state of each, manage devices and automations, provide device based presence, and surface alerts and notifications for both instances. The interface stays clean because I can only look at one instance at a time.

What you are requesting is totally different than the topic of this discussion, this change can be made in the frontend or the mobile app to allow switching between accounts but the discussion here is about linking data from entities from a client instance of HA to a main instance.

Jc2k commented 2 years ago

@blokpardi I Agree with @thecode here, I think you have misunderstood what I mean. That or I don't understand what you are trying to say 😆. I hope that what Shay has said answers your concerns, but if not let me try to say it in a clearer way, in case that is the source of the confusion.

A simple and not confusing UI for multiple HA installs is one like you've described. What you describe is what Home on iOS does too. There is a concept of multiple houses, and a selector for the current house. You are never looking at 2 houses at once. There is a single app, and it it is aware of multiple "backends". I agree that like you say, this is clean and simple. It is not a confusing mess at all.

This can be implemented by the app's already, and does not require the instances to be linked together.

A quick search suggests the maintainer for the app would actually accept the PR to implement this if somone was willing to do the work. But it is a lot of work, and they'd prefer it if the contributor was going to stick around to make sure it stayed working. They did a survey (https://twitter.com/home_assistant/status/1367226483966504960) and decided to prioritise other work themselves. See also https://github.com/home-assistant/android/issues/296, https://github.com/home-assistant/iOS/issues/161 and https://github.com/home-assistant/iOS/discussions/1790.

But all this is completely seperate to what this issue is about, where we are talking about the actual backends knowing about each other, and being able to control one from the other, entities being visible between them and events spreading between them. At a backend level, not just a UI level. For some (I think) it's about integration which is deeper than what can be done by https://github.com/custom-components/remote_homeassistant alone. For some it's about making remote_homeassistant something built in.

To get what you want you need to follow the ticket I listed above for your device of choice rather than this one, though.

blokpardi commented 2 years ago

@Jc2k thank you for the detail, that helps. My original comment was about the front-end UX and being able to manage multiple unconnected instances, so apologies if I was off topic. In terms of this thread, if you haven't seen what Adrian Caramaliu has done with WebCore (dashboard.webcore.co), it's freaking amazing IMO. It's a backend rules engine that both supports separate instances of SmartThings (technically separate instances of WebCore) but also allows those instances to talk to each other. What I wouldn't give for an open engine like that not tied to SmartThings.

ghost commented 2 years ago

I don't think we need multi HA for the apps to support multiple homes. That's a feature request for the apps.

I don't think it will ever really make sense to bolt on full and total control of one HA from another HA (like managing automations and pairing and stuff). UX wise it's quite a large thing to retrofit, short of a ground up redesign of the UI I think it would just lead to a confusing mess.

I do think one HA showing entities from another HA is interesting and useful. We can already do that to a certain extent with MQTT or HomeKit. I quite often have one HA controlling another HA by using homekit bridge on the "remote" side and homekit_controller on the "main" instance. To extend that between homes you'd just need a site to site VPN or something. It looks like the remote_homeassistant custom component might do something similar but without being limited by external schemas. All of these would let you forward just a subset of entities and events rather than forwarding everything an polluting the parent HA.

^^^^^^^^^^^^^ This realy helped me, i could not get remote home assistant working and when i used the homekitbridge function on my remote home assistant my master was easy configured with the homekit controller! :D I live on a farm and the barn has it own home assistant to measure temperature etc and i can remotely turn on heaters etc. But when im not at home only the master home assistant is connected to my phone.

john-k-mcdowell commented 2 years ago

Just adding my two cents to this thread.

I currently have HA running in my primary home and a vacation home. I believe there are several methods to address this currently available but each has limitations.

  1. Connect to one HA instance via mobile app and the other via web browser. I have two separate Nabu Casa accounts, 1 for each home. This method works, but forces the mobile phone user to know which to use for which house. (I have put a separate icon for the web version on my spouses phone. This works most of the time, but periodically causes confusion) Downside to this approach is that only one instance of HomeAssistant can use the mobile app notification engine and you can't build any "cross home" automations or scripts.

  2. Deploy remote-homeassistant and build in front end on the "master instance" the devices you want to control on the "client" instance. This works but requires building the client UIs on the Master which is ok, except I have not been able to figure out how to run a script written on the client from the master. This results in needing to duplicate all of the scripts from the client to the master for more complex UI features. Doable, just a bit of a pain.

  3. Use both methods (what I currently do). I use the remote-homeassistant so I can use "trigger" notifications and automation on the master instance with state data from the client, but use the web client to actually control the client when we are at the client home.

I am satisfied with the current state of play, except I would like to see remote-homeassistant or a similar capability added to the core.

The notifications on the Master IOS App and the ability for "cross" instance automations makes remote-homeassistant a "must" for me. The current functionality works, just not sure what the reluctance to move it into the core is. I read balloob's comments from 2018 on closing the PR. Not sure I fully understand what his concerns are, but from my perspective an MVP must include the ability to see States from the client in the Master AND call services on the client from the Master.

Hedda commented 2 years ago

I am satisfied with the current state of play, except I would like to see remote-homeassistant or a similar capability added to the core.

+1 or at least having something like remote_homeassistant or similar as a built-in core component would be a great start for this!

we are talking about the actual backends knowing about each other, and being able to control one from the other, entities being visible between them and events spreading between them. At a backend level, not just a UI level. For some (I think) it's about integration which is deeper than what can be done by https://github.com/custom-components/remote_homeassistant alone. For some it's about making remote_homeassistant something built in.

For me, to make linking two or more Home Assistant instances a good user experience it should really be a built-in core feature.

The original post by danielsjf already lists some really good real-world scenarios that should not be that uncommon today:

A few examples:

  • A house and a holiday house from the same owner
  • Two separate networks in one house
  • Bluetooth devices spread out over a larger area that cannot be connected by one Home Assistant device
  • A separate ZWave network in the pool house
  • Home Assistant runs on a device that does not have a certain connection (e.g. Bluetooth, Z-Wave, Zigbee) and a different instance of HA on a second device is needed to connect these devices

In all these cases, it would be great if two different versions of Home Assistant could talk to each other. Since one will always be the aggregator of the data, I will call this main instance of Home Assistant the master. The second device will be the client.

An additional scenario is instances of Home Assistant at parents/grandparents houses (each with Zigbee USB dongles for ZHA).

Today I manage Home Assistant at my parents via web interface only as totally separate installations with no connections at all.

genebean commented 2 years ago

As my usage has increased, I’ve found a couple of key scenarios are really hard to deal with today:

In both there it may be desirable to have devices at multiple physical locations interact with a single HA (literally or effectively) so that automations can use data from multiple places. An example of this is presence detection that knows when I’m in remote location foo instead of at home and changes behaviors of devices at both locations accordingly. Such a scenario really benefits from a single mobile app being used to provide the presence for all locations.

dinan5 commented 2 years ago

Hello:

Has anyone got Nest cameras working remotely? I have my "slave" HA running all devices that require (or benefit from) Nabu Casa and I have my master pulling data from the slave instance. Everything works beautifully, except my Nest cameras. I see "camera not found" in the master's log. I'm suspecting something with SSL but not sure.

Thanks.

bitvilag commented 1 year ago

What is the outcome? I am also interested, but I would vision a central instance for connecting the mobiles and multiple satellite instances which would work on their own isolated, but could be centrally managed if connection is reestablished, but this does mean a state replication.

Gfkappa commented 1 year ago

Hello:

I'm also interested on progress on this topic, here my usecase:

Probably Remote Home-Assistant is the solution to this case, I'll rty to give it a try but a solution at back-end level would be something really great.

Thanks,

frenck commented 1 year ago

This architecture issue is old, stale, and possibly obsolete. Things changed a lot over the years. Additionally, we have been moving to discussions for these architectural discussions.

For that reason, I'm going to close this issue.

../Frenck

ehn commented 1 year ago

@frenck What would be the best way to keep up with architectural discussions related to this?

frenck commented 1 year ago

This one is stale so closed it to clean up a little. I'm happy to re-open and move it to a discussion if you think it is viable / needs resolving.

../Frenck

ehn commented 1 year ago

I don't know how useful this discussion is, but I think the feature, in some form, would be great. Just wondering if it or something in that vein is being discussed elsewhere, in which case it would be interesting to follow the discussion.

frenck commented 1 year ago

but I think the feature, in some form, would be great

Right, but this is not a repository for tracking feature requests :)

../Frenck

elafontaine commented 6 months ago

Hi @frenck , could you enlighten me as to where this discussion should happen ? I would like to follow the conversation :).

I'm also writing here to guide fellow future client :D .

frenck commented 6 months ago

@elafontaine This architectural discussion is closed. Please use the community forums for requesting and discussing features.