Zone actuator discovery not working (`RQ|000C` not returning devices)

[trvrnrth]

In my evohome system I have 4 zones. Zone 0 uses the controller as the sensor and the other three each use a thermostat. Each zone operates in single room mode and has one (or more) HR80 TRVs.

In response to the 000C discovery requests the thermostat sensors are returned but the actuators are not which results in an incomplete system schema.

If I enable eavesdropping they are picked up (presumably via the 3150 messages) but I understand that eavesdropping is to be avoided.

Here's an example of the discovery responses received:

2024-01-01T16:55:46.795330 ... RQ --- 18:204815 01:128378 --:------ 000C 002 000F  # {'zone_type': '0F', 'domain_id': 'FC', 'device_role': 'appliance_control'}
2024-01-01T16:55:46.810380 ... RP --- 01:128378 18:204815 --:------ 000C 006 000F7FFFFFFF  # {'zone_type': '0F', 'domain_id': 'FC', 'device_role': 'appliance_control', 'devices': []}
2024-01-01T16:55:50.814342 ... RQ --- 18:204815 01:128378 --:------ 000C 002 000E  # {'zone_type': '0E', 'domain_id': 'FA', 'device_role': 'hotwater_valve'}
2024-01-01T16:55:50.828212 ... RP --- 01:128378 18:204815 --:------ 000C 006 000E7FFFFFFF  # {'zone_type': '0E', 'domain_id': 'FA', 'device_role': 'hotwater_valve', 'devices': []}
2024-01-01T16:55:51.617649 ... RQ --- 18:204815 01:128378 --:------ 000C 002 010E  # {'zone_type': '0E', 'domain_id': 'F9', 'device_role': 'heating_valve'}
2024-01-01T16:55:51.632418 ... RP --- 01:128378 18:204815 --:------ 000C 006 010E7FFFFFFF  # {'zone_type': '0E', 'domain_id': 'F9', 'device_role': 'heating_valve', 'devices': []}
2024-01-01T16:55:54.620154 ... RQ --- 18:204815 01:128378 --:------ 000C 002 000D  # {'zone_type': '0D', 'domain_id': 'FA', 'device_role': 'dhw_sensor'}
2024-01-01T16:55:54.634265 ... RP --- 01:128378 18:204815 --:------ 000C 006 000D7FFFFFFF  # {'zone_type': '0D', 'domain_id': 'FA', 'device_role': 'dhw_sensor', 'devices': []}
2024-01-01T16:56:00.465092 ... RQ --- 18:204815 01:128378 --:------ 000C 002 0208  # {'zone_type': '08', 'zone_idx': '02', 'device_role': 'rad_actuator'}
2024-01-01T16:56:00.480233 ... RP --- 01:128378 18:204815 --:------ 000C 006 02087FFFFFFF  # {'zone_type': '08', 'zone_idx': '02', 'device_role': 'rad_actuator', 'devices': []}
2024-01-01T16:56:00.654236 ... RQ --- 18:204815 01:128378 --:------ 000C 002 0108  # {'zone_type': '08', 'zone_idx': '01', 'device_role': 'rad_actuator'}
2024-01-01T16:56:00.669204 ... RP --- 01:128378 18:204815 --:------ 000C 006 01087FFFFFFF  # {'zone_type': '08', 'zone_idx': '01', 'device_role': 'rad_actuator', 'devices': []}
2024-01-01T16:56:00.855406 ... RQ --- 18:204815 01:128378 --:------ 000C 002 0308  # {'zone_type': '08', 'zone_idx': '03', 'device_role': 'rad_actuator'}
2024-01-01T16:56:00.870040 ... RP --- 01:128378 18:204815 --:------ 000C 006 03087FFFFFFF  # {'zone_type': '08', 'zone_idx': '03', 'device_role': 'rad_actuator', 'devices': []}
2024-01-01T16:56:10.498054 ... RQ --- 18:204815 01:128378 --:------ 000C 002 0204  # {'zone_type': '04', 'zone_idx': '02', 'device_role': 'zone_sensor'}
2024-01-01T16:56:10.513083 ... RP --- 01:128378 18:204815 --:------ 000C 006 0204005838A7  # {'zone_type': '04', 'zone_idx': '02', 'device_role': 'zone_sensor', 'devices': ['22:014503']}
2024-01-01T16:56:10.699081 ... RQ --- 18:204815 01:128378 --:------ 000C 002 0104  # {'zone_type': '04', 'zone_idx': '01', 'device_role': 'zone_sensor'}
2024-01-01T16:56:10.714009 ... RP --- 01:128378 18:204815 --:------ 000C 006 010400587130  # {'zone_type': '04', 'zone_idx': '01', 'device_role': 'zone_sensor', 'devices': ['22:028976']}
2024-01-01T16:56:10.899775 ... RQ --- 18:204815 01:128378 --:------ 000C 002 0304  # {'zone_type': '04', 'zone_idx': '03', 'device_role': 'zone_sensor'}
2024-01-01T16:56:10.914799 ... RP --- 01:128378 18:204815 --:------ 000C 006 0304005838F7  # {'zone_type': '04', 'zone_idx': '03', 'device_role': 'zone_sensor', 'devices': ['22:014583']}
2024-01-01T16:56:44.550485 ... RQ --- 18:204815 01:128378 --:------ 000C 002 0008  # {'zone_type': '08', 'zone_idx': '00', 'device_role': 'rad_actuator'}
2024-01-01T16:56:44.564491 ... RP --- 01:128378 18:204815 --:------ 000C 006 00087FFFFFFF  # {'zone_type': '08', 'zone_idx': '00', 'device_role': 'rad_actuator', 'devices': []}

Parsing all packets for a day with no extra config schema provided (so I expect the missing sensor for zone 0) gives me this resultant schema which is missing the actuators:

Schema[01:128378 (evohome)] = {
    "system": {
        "appliance_control": null
    },
    "orphans": [],
    "stored_hotwater": {},
    "underfloor_heating": {},
    "zones": {
        "00": {
            "_name": "Downstairs",
            "class": "radiator_valve",
            "sensor": null,
            "actuators": []
        },
        "01": {
            "_name": "REDACTED",
            "class": "radiator_valve",
            "sensor": "22:028976",
            "actuators": []
        },
        "02": {
            "_name": "Master Bed",
            "class": "radiator_valve",
            "sensor": "22:014503",
            "actuators": []
        },
        "03": {
            "_name": "REDACTED",
            "class": "radiator_valve",
            "sensor": "22:014583",
            "actuators": []
        }
    }
}

The expected schema (which I get with eavesdropping enabled) is:

Schema[01:128378 (evohome)] = {
    "system": {
        "appliance_control": "10:073263"
    },
    "orphans": [],
    "stored_hotwater": {},
    "underfloor_heating": {},
    "zones": {
        "00": {
            "_name": "Downstairs",
            "class": "radiator_valve",
            "sensor": "01:128378",
            "actuators": [
                "00:016737",
                "04:207372",
                "04:207480"
            ]
        },
        "01": {
            "_name": "REDACTED",
            "class": "radiator_valve",
            "sensor": "22:028976",
            "actuators": [
                "00:017214"
            ]
        },
        "02": {
            "_name": "Master Bed",
            "class": "radiator_valve",
            "sensor": "22:014503",
            "actuators": [
                "04:207481"
            ]
        },
        "03": {
            "_name": "REDACTED",
            "class": "radiator_valve",
            "sensor": "22:014583",
            "actuators": [
                "04:207351"
            ]
        }
    }
}

I notice that the appliance control is also not detected without eavesdropping enabled but have not investigated that issue at all. It would also appear that the eavesdropping is able to infer that zone 0 is using the controller as the sensor which is convenient (but I'm guessing unreliable?).

I can provide a config schema with the actuators listed but wanted to raise this as it didn't seem like the expected behaviour. I am of course unsure if it's an issue with my setup but I don't believe there is anything particularly odd about it.

[zxdavb]

Thanks for your rather well-structured issue! Unfortunately, my answers won't be as well organized.

would also appear that eavesdropping is ... unreliable

Eavesdropping is unsatisfactory for a few reasons, including:

• it is non-deterministic (e.g. corrupt/missing packets)
• it relies upon the non-controller devices having the same configuration as the controller - but it is quite common for them to disagree (the symptoms you describe could be an example of this)
• if people change their config, they are obligated to remember to update the config schema.

I often seriously consider removing this feature.

presumably via the 3150 messages

No, the 30C9 - I leverage the sync_cycle array that is sent every 3 minutes or so against the temps reported by devices that have temperature sensors.

Thus, this only works for zone sensors, not actuators. I simply assume that if a TRV as a zone sensor, then it is also an actuator (this is not necessarily true). IIRC, there is no reliable way to find zone actuators.

The above was only intended to find the zone that has the CTL as its sensor. It was written before I discovered/decoded the 000C command.

Knowing which sensor is configured for which zone is largely of academic interest. The CTL 'knows' the temp of each zone, regardless, and you can just ask it.

What is useful is knowing what actuators are in which zone, because that is used to calculate each zone's heat demand (unlike a zone temp, the CTL will not give you this information).

I notice that the appliance control is also not detected without eavesdropping enabled

OTBs (unlike BDRs) are never discoverable. That is, the RP payload for a RQ|CTL|000C|000F is always 000F7FFFFFFF when an OTB is the appliance controller.

Thus, the solution for this is to a) eavesdrop (see above) or use a static (hand-crafted/config) schema.

(eavesdropping the OTB I would be comfortable with)

In any case, not having your OTB in your system schema is of no consequence.

Similarly, when the CTL is a zone sensor, the RP will always have a payload of 0x047FFFFFFF: obviously, the CTL knows this information, but it simply wont tell you.

[zxdavb]

"00:016737",

Oddly, these TRVs (those that start 00:), even when correctly bound to their CTL, never appear in RP|000C payloads.

This is the only time when it is truly useful to hand-craft a system schema (as there is no discovery), so that you can get ramses_rf to calculate each zone's heat demand.

Unfortunately, there is no effective means to eavesdrop this data in all cases.

[zxdavb]

"04:207372", "04:207480"

Why theses are not being picked up? Dunno. A good first step would be to re-bind them & see what happens.

[trvrnrth]

Thanks for the detailed and helpful explanation. FWIW my 00 devices are v2.02 HR80s and the 04 devices v2.04 HR80s. I would suspect that as these are all older devices they share the same behaviour of not being returned by the controller.

For my small system maintaining a schema in config isn't really a problem as changes are very rare. Unfortunately it looks like there may be a secondary issue with loading schemas from config at present as in my testing (via ramses_cc) whilst the schema passed to the Gateway init is correct, this is not reflected in the schema later returned by the system or gateway. I have not yet had time to investigate further.

[trvrnrth]

It looks like https://github.com/zxdavb/ramses_rf/commit/e0e435d50167c0e880fedcb8f7d0cfa10ba60ca3 introduced a change whereby if cached packets are passed in on Gateway start the current state/schema is always cleared which includes any schema passed in on init and loaded here.

This will mean that changes to the schema in config are not respected if the state is being restored from ramses_cc which doesn't seem correct to me. I'm not sure I follow the intentions behind all the various load stages though so I'm unsure what the preferred fix is.

[trvrnrth]

Actually, it occurs that this will be affecting restoration of the cached schema in ramses_cc as well as any schema changes in config. So the issue is in fact that at present if ramses_cc is set to restore state (cached packets) then any cached/config/merged schema is always cleared.

[zxdavb]

I'll have a look - the set_state() method needs to go - it was a bad idea from the start.

I have tried to fix it without an extensive refactor... This project is full of issues like this - design decisions made years ago, during rapid/incremental development.

Goal would be to supply cache/schema only during init.

[trvrnrth]

👍 I'll keep an eye out for the fix.

I have tried to fix it without an extensive refactor... This project is full of issues like this - design decisions made years ago, during rapid/incremental development.

The joys of reverse engineering and backwards compat 😆

[trvrnrth]

Resolved with the 0.31.1 release so closing this now 🎉