Only detecting device after power cycle? Not saving devices.json correctly?

[samus-homebridge]

Hello, and thanks for a very helpful project!

I have a house that used to have a Honeywell ST9400C controller with a CS92A DHW thermostat on the hot water tank. I recently replaced the ST9400C with a Nest because I need to have remote access. However, I've been working on getting evoGateway up and running to be able to read the hot water temp.

When I first run the script, it enters eavesdropping mode but doesn't detect anything, even after an hour or more. However, if I power cycle the CS92A by popping out the batteries and back in, there is an immediate surge of registration messages followed by reporting of the hot water temperature every minute.

If I then close evoGateway, it reports that an updated devices.json and ramses_rf_schema.json have been created.

However, devices.json does not list the DHW sensor that it had just detected - just evoGateway and a controller. The schema file likewise just reports the controller, not the sensor.

If I then restart evoGateway, first it reports that no devices file was found so it is re-entering eavesdropping mode and sends a long series of pink control messages, but no reporting from the sensor.

If I manually edit devices.json to add a new json entry with the ID that was detected in the first try above with name DHW, and then re-run, it still says "Existing devices file not found" for some reason, but my manually entered sensor appears on the allow_list and it reports the temperature every minute again.

So I have a couple of questions really: 1) why is the device only being detected after a power cycle? I'm concerned because in the course of testing it last night, at some point it stopped detecting the sensor signals until I started all over again, which won't be possible when we are away from time? 2) What's up with the partial failure to detect the devices.json file? On the latter point the relevant if statement is at line 949 of evogateway.py, so one or both of

if "schema" in DEVICES and "zones" in DEVICES["schema"]: is false - probably because I manually edited devices.json? but I am not sure what the correct format should be. Thanks for any pointers!

[smar000]

1) why is the device only being detected after a power cycle? I'm concerned because in the course of testing it last night, at some point it stopped detecting the sensor signals until I started all over again, which won't be possible when we are away from time? 2) What's up with the partial failure to detect the devices.json file?

I haven't experience this myself and have a DHW sensor in my system too. However, on thinking about, I have a suspicion that this may be due to a recent change I introduced to address some of the underlying ramses_rf changes in recent updates. I'll have a look at it as soon as I get some time, and will get back to you.

[samus-homebridge]

That's great, thank you! As an additional data point, when I started from scratch earlier this afternoon, closed evoGateway and then restarted it within a few minutes it reported the DHW temps with no problem. But then I closed it again, went out for about 3 hours and just re-ran evoGateway and it's not reporting it any more (though the sensor remains on allow_list). Of course happy to post any output that could be useful to you.

[samus-homebridge]

One more data point that might be helpful. When the DHW sensor is not reporting to evoGateway, pressing the button on the transceiver turns on constant red, indicating comms failure (not really a surprise). Just now when I had power-cycled again and pressed the button, it came on green indicating good comms with the evohome controller, but evoGateway recorded the following:

2021-07-11T17:30:52.999196 049 RQ --- 07:008654 23:034736 --:------ 0016 002 0100 < Corrupt packet: Invalid code for 23:034736 to Rx: 0016 (will be ignor
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/ramses_rf/message.py", line 445, in is_valid
    self._payload = payload_parser(self.raw_payload, self)
  File "/usr/local/lib/python3.7/dist-packages/ramses_rf/parsers.py", line 268, in wrapper
    check_verb_code_dst(msg)
  File "/usr/local/lib/python3.7/dist-packages/ramses_rf/parsers.py", line 234, in check_verb_code_dst
    raise CorruptPacketError(f"Invalid code for {msg.dst.id} to Rx: {msg.code}")
ramses_rf.exceptions.CorruptPacketError: Corrupt packet: Invalid code for 23:034736 to Rx: 0016 (will be ignored)

Which kept repeating until I hit the button again, the green light went out on the transceiver and it went back to reporting temps to evoGateway. (07:008654 is the device ID for the sensor and 23:034736 the ID for the controller).

[smar000]

The lack of DHW temperature data is unlikely related to evoGateway - all packet receipt/processing is being done by ramses_rf library now. However, this issue of infrequent DHW temperature messages is a known issue in the community, and is believed to be a hardware issue. If you do a search, you will find references to it. I myself had a DHW wireless sender replaced under warranty for this type of issue (my original sender worked fine for many months, and then one day started sending updates only every hour or so).

Just now when I had power-cycled again and pressed the button, it came on green indicating good comms with the evohome controller, but evoGateway recorded the following:

2021-07-11T17:30:52.999196 049 RQ --- 07:008654 23:034736 --:------ 0016 002 0100 < Corrupt packet: Invalid code for 23:034736 to Rx: 0016 (will be ignor
Traceback (most recent call last):

The above error is again entirely as reported by ramses_rf and not from evoGateway. I suggest checking with David, the author of ramses_rf, if he has any insights into this.

Finally, I have just now uploaded a newer version which should hopefully address the schema load issues on initial/clean startups etc.

[samus-homebridge]

Thanks! There is definitely some improvement, but possibly still an issue.

When I ran it for the first time, it successfully detected the DHW sensor after it broadcast its temp (once per hour).

On closing, it saved a new devices.json containing that sensor plus the evoGateway itself. However the ramses_rf_schema now contained only the DHW sensor and not the "type 23" controller that had been in there previously.

I'm not sure if that is a problem or not, but when re-running evoGateway with those files it gave:

2021-07-12 15:23:44,395 [913] The existing schema file 'ramses_rf_schema.json' appears to be invalid. Ignoring...
2021-07-12 15:23:44,395 [935] Schema file missing or the 'allow_list' section is missing. Defaulting to ramses_rf 'eavesdropping' mode

before loading the two devices (DHW and evoGateway "18:" device).

Edit: also to note that in the previous version the DHW sensor was correctly added to the allow_list notwithstanding that it wasn't saved to devices.json, but in the current version it reported that no allow_list was configured.

[smar000]

However the ramses_rf_schema now contained only the DHW sensor and not the "type 23" controller that had been in there previously.

The script only saves whatever it has from the ramses_rf Gateway object. As it posts an updated schema to the mqtt broker each time it detects a new device, can you check there (before exiting the system) to see what the current state of the schema looks like, and if the device (type 23) you are looking for is there? You can also use this approach to know how long to keep the script running the first time until all your devices are found.

I'm not sure if that is a problem or not, but when re-running evoGateway with those files it gave:

2021-07-12 15:23:44,395 [913] The existing schema file 'ramses_rf_schema.json' appears to be invalid. Ignoring...
2021-07-12 15:23:44,395 [935] Schema file missing or the 'allow_list' section is missing. Defaulting to ramses_rf 'eavesdropping' mode

before loading the two devices (DHW and evoGateway "18:" device).

This is sort of by design! Basically if there is no controller id in the schema file, it ignores the whole schema file and goes into eavesdropping mode so as to re-discover all the devices/schema. This would probably also explain why your allow_list is showing as not configured.

[samus-homebridge]

I see, OK. Well I'm a little more confused now because the only evohome device I have is the orphaned DHW sensor - I had assumed that the 23: controller that evoGateway was originally detecting was some sort of dummy controller built into the script, but perhaps not... In fact I don't even see type 23 as a device type listed in the Ramses II wiki so I don't know what that was.

Ultimately though I think it will be fine for my purposes because I'm just looking to observe the tank temp and relay it to Twitter using mqttwarn. So eavesdropping will be OK. Thanks again for your help and for this project!

[smar000]

Thanks for the update. I'll think about removing the controller check in the schema load routine or maybe add an option to ignore the schema file.

Anyway, If you have no other problems, I will close the issue for now.

[samus-homebridge]

No, thanks again very much!