kingsleyadam / local-abbfreeathome

A Python Package for interacting with the ABB (Busch Jaeger) Free@Home System via the Local API
MIT License
4 stars 2 forks source link

Handling of virtual devices need to be modified #134

Open derjoerg opened 1 week ago

derjoerg commented 1 week ago

I just had time to start with some more tests and - well - I think there is a problem with virtual device handling:

Regarding the ABB documentation:

Please remember that input and output datapoints are named from the perspective of the device. This means that the API user has to write to the output datapoints that are the feedback datapoints to show the current state of the virtual device. The input datapoints for a virtual device are used to control it and change its current state. The owner of the virtual device has to monitor the input datapoints for its virtual devices to react to these changes and use it to control the hardware device.

The handling of virtual devices needs to be rethinked completely. You can easily test it by creating a virtual SwitchActuator (best with ttl = -1, so you don't need to think about lifetime). If you change it in F@H, shortly after it switches back, same in HA.

This needs further investigation, but for now I would propose to remove the virtual interface support in the HA-integration.

kingsleyadam commented 1 week ago

It seems to act more of a push button than a actual switch actuator. But it still gets the FID_SWITCH_ACTUATOR function. This would be much easier to implement if they'd give it a different function id, because it actually functions differently.


      "6000D2CB27B2": {
        "floor": "01",
        "nativeId": "abcd12345",
        "room": "18",
        "deviceId": "0001",
        "displayName": "Virtual switch",
        "unresponsive": true,
        "unresponsiveCounter": 3,
        "defect": false,
        "channels": {
          "ch0000": {
            "floor": "01",
            "room": "18",
            "displayName": "Virtual switch",
            "selectedIcon": "1",
            "functionID": "7",
            "inputs": {
              "idp0000": {
                "pairingID": 1,
                "value": "1"
              },
              "idp0001": {
                "pairingID": 2,
                "value": ""
              },
              "idp0002": {
                "pairingID": 3,
                "value": ""
              },
              "idp0003": {
                "pairingID": 4,
                "value": ""
              },
              "idp0004": {
                "pairingID": 6,
                "value": ""
              }
            },
            "outputs": {
              "odp0000": {
                "pairingID": 256,
                "value": "0"
              },
              "odp0001": {
                "pairingID": 257,
                "value": "0"
              }
            },
            "parameters": {
              "par0015": "60",
              "par0014": "1"
            }
          }
        },
        "parameters": {}
      },
kingsleyadam commented 1 week ago

I also can't seem to figure out how to assign the virtual device to a scene. I don't see an option in the free@home interface.

derjoerg commented 1 week ago

I did some tests with a virtual SwitchActuator, Node-Red and a binary_input helper in HA.

Two use-cases need to be considered:

  1. Changes to a VD from F@H app
  2. Changes to a VD from HA

Regarding 1.:

Regarding 2.:

derjoerg commented 1 week ago

Let me write it in other words (I need this a bit to get clearer on that topic):

The same applies to off

Does this sounds meaningful?

kingsleyadam commented 1 week ago

Got it, that makes more sense. These VirtualDevices are really designed to implement external devices that don’t have native support within Free@Home. If you’re using Home Assistant I wonder what you’re use case for a Virtual Device would then be as you’d just do it all in Home Assistant, which has better support for a number of platforms.

They’re requirement for a Virtual Device then seems like a bit of a hack in order to trigger a scene via the API (which I still haven’t figured out how to hook-up yet).

derjoerg commented 1 week ago

Got it, that makes more sense. These VirtualDevices are really designed to implement external devices that don’t have native support within Free@Home. If you’re using Home Assistant I wonder what you’re use case for a Virtual Device would then be as you’d just do it all in Home Assistant, which has better support for a number of platforms.

My use-case is the following:

I can control this through my HA dashboard. But I also have a F@H panel (SmartTouch 10') and for my wife and my kids to make it easier, I created two virtual switch actuators and put them on the panel, which is placed in our living-room. So they can just tip on the appropriate icon on the panel to turn on/off the automations.

derjoerg commented 1 week ago

They’re requirement for a Virtual Device then seems like a bit of a hack in order to trigger a scene via the API (which I still haven’t figured out how to hook-up yet).

You can create in F@H an action (I hope this is the correct translation), as event you use the virtual device turn on and as action you can select the scene.

kingsleyadam commented 1 week ago

You can create in F@H an action (I hope this is the correct translation), as event you use the virtual device turn on and as action you can select the scene.

Ok this is progress, that'll work for triggering this scene from Home Assistant. But I also wanted to know when his scene was triggered by Free@Home so I can have Home Assistant do some additional steps (e.g. turn ventilation unit to low). If they would just send any scene's being run as a trigger over the web socket that'd make everything much easier. I'm not sure why they don't do that.

Instead it seems like I have to expose all those additional devices from Home Assistant that I want in the scene as Virtual Devices in Free@Home and then include them in the scene.

kingsleyadam commented 1 week ago

My use-case is the following:

  • In HA I have defined a sleep-mode an night-mode automation. At the moment they are triggered through a binary-input-helper

    • If I turn on sleep-mode the covers in the nursery close and the lights go on dimmed
    • If I turn off sleep-mode the lights in the nursery go off
    • If I turn on night-mode all covers in my home close, several RTCs go to ECO mode and some electrical consumers go off
    • If I turn off night-mode all covers go up, the RTCs return to "normal" mode and the electrical consumers go on

I can control this through my HA dashboard. But I also have a F@H panel (SmartTouch 10') and for my wife and my kids to make it easier, I created two virtual switch actuators and put them on the panel, which is placed in our living-room. So they can just tip on the appropriate icon on the panel to turn on/off the automations.

This makes a lot of sense, so in this case you want to control Home Assistant from Free@Home. Just curious, could you do all of these same things as a scene in Free@Home :)?

I think we can add virtual device support without having to rearchitect anything. The classes we've already built will have a new attribute "is_virtual_device" and will have additional ack/acknowledgement methods that'll send a message back to Free@Home when certain inputs are read from the websocket.

When it comes to turning On/Off the switch from Home Assistant, we'd just have to change from sending the input to output if it's a virtual device.

derjoerg commented 1 week ago

You can create in F@H an action (I hope this is the correct translation), as event you use the virtual device turn on and as action you can select the scene.

Ok this is progress, that'll work for triggering this scene from Home Assistant. But I also wanted to know when his scene was triggered by Free@Home so I can have Home Assistant do some additional steps (e.g. turn ventilation unit to low). If they would just send any scene's being run as a trigger over the web socket that'd make everything much easier. I'm not sure why they don't do that.

Instead it seems like I have to expose all those additional devices from Home Assistant that I want in the scene as Virtual Devices in Free@Home and then include them in the scene.

Have you looked here https://developer.eu.mybuildings.abb.com/fah_local/concepts? Scroll down to "Scene support for virtual devices", perhaps this might help you.

I have to admit, that I don't use scenes and actions in F@H. I try to keep F@H as dump as possible and do everything advanced in HA. This is my personal decision as I only want to look in one system if I need to make changes and don't have to think about ("In which system is this functionality implemented?"). Sure, the basic stuff is done in F@H like linking rockers to lights and RTCs to heat valves, but I try to keep the rest completely in HA. That's why I use the virtual devices as explained above in my use-case.

derjoerg commented 1 week ago

This makes a lot of sense, so in this case you want to control Home Assistant from Free@Home. Just curious, could you do all of these same things as a scene in Free@Home :)?

😄 As just written in another comment: I would be able to control most of the stuff also with scenses in F@H, but not everything (e.g. turn of POE-ports on my switch, send out statistics regarding my daily PV-usage and so on). Sure, I can split things up, do part of it through scenes in F@H and part in HA, but - besides the problem you already mentioned, that it is not easily possible to find out if a scene was triggered - I would always, when I want to do a modification, check if it needs to be done in F@H or in HA.

I think we can add virtual device support without having to rearchitect anything. The classes we've already built will have a new attribute "is_virtual_device" and will have additional ack/acknowledgement methods that'll send a message back to Free@Home when certain inputs are read from the websocket.

When it comes to turning On/Off the switch from Home Assistant, we'd just have to change from sending the input to output if it's a virtual device.

This sounds amazing. If this would be a solid implementation I would be able to get rid of my NodeRed workaround and other virtual devices would become interessting, e.g.:

kingsleyadam commented 1 week ago

Have you looked here https://developer.eu.mybuildings.abb.com/fah_local/concepts? Scroll down to "Scene support for virtual devices", perhaps this might help you.

Yea, I looked through it. But this also doesn't really resolve it. I"m not sure I understand the use case of scenesTriggered since it's going to send the updated output for each device in the state separately via the websocket anyway. The scenesTriggered just tells me all of the devices in the scene and it's output state. It doesn't actually tell me which scene was triggered.

kingsleyadam commented 1 week ago

Ok, so this is strange. I don't get the interface at all for VirtualDevices. Although the documentation clearly states otherwise. Do you?

Another attribute to identify virtual device in the data model is the interface attribute whose value have a prefix of "vdev:".

      "6000D2CB27B2": {
        "floor": "01",
        "nativeId": "abcd12345",
        "room": "18",
        "deviceId": "0001",
        "displayName": "Virtual switch",
        "unresponsive": true,
        "unresponsiveCounter": 3,
        "defect": false,
        "channels": {
          "ch0000": {
            "floor": "01",
            "room": "18",
            "displayName": "Virtual switch",
            "selectedIcon": "1",
            "functionID": "7",
            "inputs": {
              "idp0000": {
                "pairingID": 1,
                "value": "1"
              },
              "idp0001": {
                "pairingID": 2,
                "value": ""
              },
              "idp0002": {
                "pairingID": 3,
                "value": ""
              },
              "idp0003": {
                "pairingID": 4,
                "value": ""
              },
              "idp0004": {
                "pairingID": 6,
                "value": ""
              }
            },
            "outputs": {
              "odp0000": {
                "pairingID": 256,
                "value": "0"
              },
              "odp0001": {
                "pairingID": 257,
                "value": "0"
              }
            },
            "parameters": {
              "par0015": "60",
              "par0014": "1"
            }
          }
        },
        "parameters": {}
      },
derjoerg commented 1 week ago

Yes

      "60005D808C54": {
        "floor": "02",
        "nativeId": "virtual-switch-sleep",
        "room": "06",
        "interface": "vdev:installer@busch-jaeger.de",
        "deviceId": "0001",
        "displayName": "Schlafmodus",
        "unresponsive": false,
        "unresponsiveCounter": 0,
        "defect": false,
        "channels": {
          "ch0000": {
            "displayName": "Schlafmodus",
            "floor": "02",
            "function": "FID_SWITCH_ACTUATOR",
            "functionID": "7",
kingsleyadam commented 1 week ago

Yes

      "60005D808C54": {
        "floor": "02",
        "nativeId": "virtual-switch-sleep",
        "room": "06",
        "interface": "vdev:installer@busch-jaeger.de",
        "deviceId": "0001",
        "displayName": "Schlafmodus",
        "unresponsive": false,
        "unresponsiveCounter": 0,
        "defect": false,
        "channels": {
          "ch0000": {
            "displayName": "Schlafmodus",
            "floor": "02",
            "function": "FID_SWITCH_ACTUATOR",
            "functionID": "7",

Ok, this is problematic. It's not coming through for me at all. I've tried multiple virtual devices. We won't be able to implement this if the interface value isn't coming through. Looks like a bug.

Can you confirm the firmware you're running on the SysAP? I'm running 3.4.0-13310

"6000EBE59C03": {
        "floor": "01",
        "nativeId": "AllOffVirtualSwitch",
        "room": "18",
        "deviceId": "0001",
        "displayName": "Everything Off Virtual Switch",
        "unresponsive": false,
        "unresponsiveCounter": 0,
        "defect": false,
        "channels": {
          "ch0000": {
            "floor": "01",
            "room": "18",
            "displayName": "Everything Off Virtual Switch",
            "selectedIcon": "1",
            "functionID": "7",
            "inputs": {
              "idp0000": {
                "pairingID": 1,
                "value": ""
              },
              "idp0001": {
                "pairingID": 2,
                "value": ""
              },
              "idp0002": {
                "pairingID": 3,
                "value": ""
              },
              "idp0003": {
                "pairingID": 4,
                "value": ""
              },
              "idp0004": {
                "pairingID": 6,
                "value": ""
              }
            },
            "outputs": {
              "odp0000": {
                "pairingID": 256,
                "value": "0"
              },
              "odp0001": {
                "pairingID": 257,
                "value": "0"
              }
            },
            "parameters": {
              "par0015": "60",
              "par0014": "1"
            }
          }
        },
        "parameters": {}
      },
derjoerg commented 1 week ago

How have you received this info? I tested the configuration- and device-api endpoint and both return it.

Yes, I have 3.4.0-13310 running

kingsleyadam commented 1 week ago

How have you received this info? I tested the configuration- and device-api endpoint and both return it.

Yes, I have 3.4.0 running

Same, both the configuration and device endpoint. Here's the device endpoint.

{
  "00000000-0000-0000-0000-000000000000": {
    "devices": {
      "6000EBE59C03": {
        "floor": "01",
        "nativeId": "AllOffVirtualSwitch",
        "room": "18",
        "deviceId": "0001",
        "displayName": "Everything Off Virtual Switch",
        "unresponsive": false,
        "unresponsiveCounter": 0,
        "defect": false,
        "channels": {
          "ch0000": {
            "floor": "01",
            "room": "18",
            "displayName": "Everything Off Virtual Switch",
            "selectedIcon": "1",
            "functionID": "7",
            "inputs": {
              "idp0000": {
                "pairingID": 1,
                "value": ""
              },
              "idp0001": {
                "pairingID": 2,
                "value": ""
              },
              "idp0002": {
                "pairingID": 3,
                "value": ""
              },
              "idp0003": {
                "pairingID": 4,
                "value": ""
              },
              "idp0004": {
                "pairingID": 6,
                "value": ""
              }
            },
            "outputs": {
              "odp0000": {
                "pairingID": 256,
                "value": "0"
              },
              "odp0001": {
                "pairingID": 257,
                "value": "0"
              }
            },
            "parameters": {
              "par0015": "60",
              "par0014": "1"
            }
          }
        },
        "parameters": {}
      }
    }
  }
}
derjoerg commented 1 week ago

This makes absolutely no sense :cry:

Your other devices have an interface entry?

kingsleyadam commented 1 week ago

This makes absolutely no sense 😢

Your other devices have an interface entry?

Yep, I was just trying to implement a virtual device in the library and was going crazy because it kept returning none when I filtered by the VIRTUAL_DEVICE function.

derjoerg commented 1 week ago

Sorry to say, but: Have you tried to reboot your SysAP?

kingsleyadam commented 1 week ago

Sorry to say, but: Have you tried to reboot your SysAP?

No, and I don't really think this should be the solution. We may have to somehow report this as a bug to ABB.

derjoerg commented 1 week ago

No, and I don't really think this should be the solution. We may have to somehow report this as a bug to ABB.

:rofl: Sorry, but good luck. Till now I already wrote several emails to them and NEVER got a single reply :cry:

kingsleyadam commented 1 week ago

@derjoerg , here's an initial branch with working code. I also noticed the free@home websocket does send an output after you acknowledge the change, so this should work. But if wanted we could add more logic to the ack function to set the state of the switch actuator, that's probably more ideal than waiting for another response from the websocket.

https://github.com/kingsleyadam/local-abbfreeathome/compare/main...u/kingsleyadam/addVirtualDeviceSupport

As you can see I had to hard code my device serial id to test. Maybe this is something you can build on? I need to focus my (larger) efforts elsewhere in life. :)

INFO:abbfreeathome.devices.base:Everything Off Virtual Switch received updated data: 6000EBE59C03/ch0000/odp0000: 1
DEBUG:abbfreeathome.api:Websocket Response: {'datapoints': {'6000EBE59C03/ch0000/idp0000': '0'}, 'parameters': {}, 'devices': {}, 'devicesAdded': [], 'devicesRemoved': [], 'scenesTriggered': {}}
INFO:abbfreeathome.devices.base:Everything Off Virtual Switch received updated data: 6000EBE59C03/ch0000/idp0000: 0
DEBUG:abbfreeathome.api:Websocket Response: {'datapoints': {'6000EBE59C03/ch0000/odp0000': '0'}, 'parameters': {}, 'devices': {}, 'devicesAdded': [], 'devicesRemoved': [], 'scenesTriggered': {}}
derjoerg commented 1 week ago

I'm just thinking out loud a different concept regarding virtual devices: As virtual devices act as a kind of proxy device between F@H and a remote system (HA) perhaps we should not go the way and add virtual devices in HA as it is not the concept. Shouldn't it be the following:

  1. For a sensor we only need to meaningful implement actions in HA to update specific specific datapoints (e.g. switch on, temperature, ...)
  2. For an actuator we just need to fire an event on the HA event bus

In HA we have e.g. a binary_input-helper (or a template-switch), which is our "real" device from a F@H perspective. If the device is switched on we need to fire the action to update the correct datapoint. If the actuator changes in F@H and automatically an event is fired we need to create an automation in HA, which listens to the event and set our device in HA accordingly.

Sure, this is much more work but would result in a lot of possibilities and it is an advanced concept

kingsleyadam commented 1 week ago

To me, the "real" device is the Home Assistant entity that this (corresponding) integration creates. And the job of this code is to ensure that Home Assistant entity's state gets updated when requested by free@home. If it updates the Home Assistant entities state it'll send a message back to Free@Home that the state is been updated accordingly. I don't think (for this code and HA integration) that scope should go any further.

You could then setup an automation that'll go off when the state of the entity is changed to update additional entities in Home Assistant as needed. I actually don't think this is too different than the event scenario you just proposed.

On another note, a lot of what I did in the linked PR could probably be moved to the Home Assistant integration as it should be up to the platform using this code to update their own device state and acknowledge the change.

An example is if I created some code to integrate physical Hue lights via Free@Home and I used this python code and virtual devices. It should be up to me to get the state of the change from the websocket, update my Hue light, then acknowledge I updated the light via an instance of the SwitchActuator class. The SwitchActuator class can provides the methods of acknowledgment, but probably shouldn't be the one acknowledging anything based on web socket responses.

derjoerg commented 1 week ago

Well, I would say that in this case I dis-agree with you. A real device in F@H is a rocker, a light, a sensor and the HA integration creates the necessary entities for them.

A virtual device is NOT a real in F@H, so the HA integration should not create such entities.

For sure, the update of the virtual datapoints can also be done from HA by creating the necessary REST calls but the retrieval of a websocket message I don't know how to do that directly in HA.

Let me explain some use cases:

derjoerg commented 1 week ago

For the actions we can look at: https://developers.home-assistant.io/docs/device_automation_action

And instead of firing events triggers can be defined: https://developers.home-assistant.io/docs/device_automation_trigger

kingsleyadam commented 1 week ago

Thanks for providing the examples and links to the actions/triggers in Home Assistant. I think that makes a lot of sense.

Just to confirm, you wouldn't make any changes to the existing device classes right? Instead we'll just provide the "highway" for which these events can travel. Does that sound correct?

That works for me. My only hesitation is because we'll depend on the user to create the ack feedback loop in the form of automations I can see a lot of user issues getting this working properly. This should be very well documented and even after that I assume some additional support will need to be provided by way of GitHub issues.

I see a bit more work involved with getting this setup. For each VirtualDevice function that we want to support we'll need to know the list of possible datapoints as actions and events the websocket should be listening on.

derjoerg commented 1 week ago

Just to confirm, you wouldn't make any changes to the existing device classes right? Instead we'll just provide the "highway" for which these events can travel. Does that sound correct?

Yes, I think we need to have for VDs a separate class-hierachy as with the current base class (but I might be wrong). The VDs would not be returned from a e.g. get_devices_by_class(device_class=SwitchActuator). It would need something different like get_virtual_devices_by_class(device_class=SwitchActuator).

So we would be able to follow the same logic like today with the base devices and implement it use-case by use-case.

That works for me. My only hesitation is because we'll depend on the user to create the ack feedback loop in the form of automations I can see a lot of user issues getting this working properly. This should be very well documented and even after that I assume some additional support will need to be provided by way of GitHub issues.

Yes, I also see this. The documentation needs to be very clear that this is an advanced topic as e.g. the user also needs to directly interact with the F@H REST-api to create the VD. But it also would be a unique feature I'm not aware of any other implementation supporting this

I see a bit more work involved with getting this setup. For each VirtualDevice function that we want to support we'll need to know the list of possible datapoints as actions and events the websocket should be listening on.

Like today with the base devices :smiley:

derjoerg commented 1 week ago

And here a possible stub for a sensor-class:

class BrightnessSensor(VirtualDevice):

    def set_brightness(self, value: float):
        await self._set_brightness_datapoint(str(value)):

    def _set_brightness_datapoint(self, value: str):
        _brightness_output_id, _brightness_output_value = self.get_output_by_pairing(
            pairing=Pairing.AL_BRIGHTNESS_LEVEL
        )
        return await self._api.set_datapoint(
            device_id=self.device_id,
            channel_id=self.channel_id,
            datapoint=_brightness_output_id,
            value=value,
        )

But perhaps it can also derive from the current base class

In HA this would result in an action: "ABB Free@Home: MyWeatherStation: Set Brightness"

derjoerg commented 4 days ago

@kingsleyadam Anything I can help with?

kingsleyadam commented 3 days ago

I don’t really have the capacity to implement virtual devices. When I do have some time again I’ll be focusing on the HA integration and getting a PR together for HA Core.

I’ll put this into the backlog for now.

derjoerg commented 3 days ago

OK, understand.

I will try to come around with a possible implementation and then we can see

derjoerg commented 3 days ago

Ok, so this is strange. I don't get the interface at all for VirtualDevices. Although the documentation clearly states otherwise. Do you?

      "6000D2CB27B2": {

Just as an FYI:

After some tests I now have the same behavior :rofl:

I'm switching now from checking the interface-attribute to checking if the left four characters of the device_id are '6000'