Opentherm Gateway doesn‘t update values

[umtauscher]

The problem

After the initial connect Opentherm Gateway disconnect and doesn‘t do anything any more. After Restarting the integration it works once an then stops again.

Ot-Monitor doesn‘t show any open connections.

What version of Home Assistant Core has the issue?

2022.7.6

What was the last working version of Home Assistant Core?

don‘t know

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Opentherm Gateway

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[probot-home-assistant[bot]]

opentherm_gw documentation opentherm_gw source _{^{(message by IssueLinks)}}

[probot-home-assistant[bot]]

Hey there @mvn23, mind taking a look at this issue as it has been labeled with an integration (opentherm_gw) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[mvn23]

Thank you for the report. Unfortunately I can't reproduce the issue, so I will need some more info to get started. How are you connecting to the gateway? Via USB/serial or network? You mentioned otmonitor. Is otmonitor connected to the gateway at the same time as Home Assistant? Can you provide a full debug log of a connection being made up to the point where it does not update anymore?

[umtauscher]

Wow, thank you for you super speedy raktion.

I am connecting through a socket connection to an OT-Monitor instance that runs on a Rpi connected to the otgw via USB. I have activated the relay server and relay opentherm messages on port 7686. At the moment OT Monitor shows 1 active connection to my HA instance :52882 The last log entry on HA is one hour old stating "changed to heating" which is always the only entry, Thank you very much for your effort. Cheers

[umtauscher]

Please don't mind my stupid question, but how would I produce a full debug log for Opentherm gateway?

[umtauscher]

otgw_debug.log

[mvn23]

The last log entry on HA is one hour old stating "changed to heating" which is always the only entry

This is quite common on most northern European household systems which can only affect the temperature by heating. They never switch to cooling or any other mode, which is what is logged by Home Assistant in the history.

I see in your log (thank you for that) that the updates are being received as they should. Can you test a round-trip communication to the gateway? A quick way of doing that is to open the climate entity in the UI to the Details tab, change the Target Temperature so it becomes red and see if the temperature in the top-right of that same dialog changes along with it (may take 1-2 seconds). If this works then the connection to the gateway is OK and updates are processed as they should.

If that works, could you provide some more details on what doesn't work?

[umtauscher]

My thermostat doesn't alow to set an override temperature. I can see, that the messages are received in the log. Yet every sensor that I look at, has a timestamp "30 minutes ago" - when I last restarted. Maybe the messages are received but not updated in the UI?

[mvn23]

The sensors are only updated when the value actually changes. Most values remain constant over time, except for some of the temperatures but those may be affected by rounding. You can try to find a value which does change significantly (or one that you can force to change) like the Room Setpoint (Thermostat) or the Boiler Flame (Boiler) and see if that updates correctly. The former should update whenever you change the desired temperature on your thermostat, the latter when you use hot water from the boiler, either for central heating or for domestic hot water.

Things like the room setpoint may take a while to propagate. Please wait 1-2 minutes after changing the setting to see if the sensor changes.

[umtauscher]

Ok, that seems to work. I am very sorry if I reported a non existant bug. I will report back tomorrow about the outcome. Thanks again for your help. Cheers

[umtauscher]

I just checked. The time doesn't change in a sensor, when the value is updated. I suppose that is normalö behaviour?

[mvn23]

Do you mean the "last updated" time for the sensor as in the attached screenshot? That's not something the OpenTherm Gateway integration controls and it should be handled automatically.

[umtauscher]

Yes, that "timestamp" only changes here, once when I restart the integration or HA, not when the sensor changes value.

[umtauscher]

What precision have you set? I have 0.1 for both, no override setpoint

[mvn23]

I left them both at 0 (defaults to 0.5 for °C), but the separate room temp sensor in the screenshot uses 0.1.

As I said, the integration does not control the last updated timestamp. If it does not update then your other non-OpenTherm sensors should be affected as well. If this is the case then please open a new bug report for that issue as it is not related to the OpenTherm Gateway integration. For debugging, you can see the raw status updates by listening for the state_changed event in the developers tools.

[umtauscher]

Thanks, I missed the "NOT" in you last explanation....

[umtauscher]

Other sensors are not affacted - at least I see none. But if it helps, I will open a new bug report. Thanks

[mvn23]

If only OpenTherm entities are affected then a new bug report won't be necessary. Just to be clear: which sensor (entity) is not updating its last update time when it gets a new value? For the climate entity this is expected behavior as long as it remains in heat mode, but all sensor and binary_sensor entities should work.

[umtauscher]

I don't see any of them updating the time. This goes for the control as well, not only sensors. At the moment I have the following activated: climate.otgw1 sensor.outside_temp_thermostat_otgw1 sensor.ch_water_temp_thermostat_otgw1 sensor.room_temp_boiler_otgw1 sensor.dhw_setpoint_thermostat_otgw1 sensor.dhw_temp_thermostat_otgw1

I can see i.e. Climate (temperature) changing, when I turn the knob on the thermostat (that's what you sugessted to test) but the time of last change stays the same.

[mvn23]

Ok, for the climate entity this is expected behavior. That one will only change the update time if the mode (heat) changes. This is a design choice made in Home Assistant. If you enable the Room Setpoint (Thermostat) sensor and turn the knob, does that sensor update correctly?

[umtauscher]

Yes it does. Am I chasing ghosts here?

[mvn23]

I'm sorry to say, but I guess so 🤣

If everything works now as designed (although maybe not entirely as expected), please consider closing the issue.

[umtauscher]

Ok. Thank you very much for your patience and kindness. Have a nice day!

[umtauscher]

Unfortunately I have to reopen this. Since yesterday the last sensor that has been updated was 16 hours ago. As I see it, it took openntherm gateway to finally close until this morning.

otgw_debug.log

Since 16 hours all sensor stay static/do not get new data - its not only the timestamp. Cheers

[mvn23]

Thanks for the log. It looks like you found a bug in the reconnect logic of pyotgw. I will fix it there over the next days and then update the dependency in Home Assistant.

[umtauscher]

I have updated the OS to 8.4 an Core to 2022.7.7 about 2 hours ago. Sadly I have to report, that none on my sensors have been updated ever since. So the problem is not resolved. Sorry

[umtauscher]

to add a fact, OTGW outside temperatur dropped to 20.08 C since, it still is 20.9 in homeassistant.

[umtauscher]

I deleted the last message because it was wrong. My ipad fooled me while reading the log... ;-)

[mvn23]

Can you provide debug logs of either an integration reload or HA start?

[umtauscher]

This is the complete log since the update home-assistant.log

And here is a screenshot of a sensor. (They all look the same)

Hope this helps

[mvn23]

It does help indeed. It looks like (in this specific case) you are using the Hot Water Temperature (Thermostat) sensor, which should and does show you the hot water temperature that your thermostat is aware of at the moment. The OpenTherm Gateway is smart enough to optimize its communication to both the boiler and the thermostat. In your case, it concluded that your thermostat has no use for the Hot Water Temperature, so it stopped forwarding the related messages. This is why the (Thermostat) sensor stopped updating. However, the gateway is still receiving the information from the boiler. The climate entity takes its data from the most appropriate source automatically. If you need to enable additional sensors, please make sure you take them from the appropriate source. In this case, the boiler is reporting the Hot Water Temperature, so you will most likely want the Hot Water Temperature (Boiler) sensor. According to your debug log, that one should be updating as expected.

[umtauscher]

Thank you for your long explanation. But I think smart is the wrong word for what is going on. How should someone - a user - determin what entities to use if they are populated in the beginning with real values. After that "smart" decision suddenly they ar not updated? That is simply not user friendly. If they are updated on start, they should be maintained afterwards too. Or a suggestion: Why not activate the used entities from the start and keep the others deactivated?

Anyways, three hours later, the device decided not to be available any more. Any attempt to reactivate it, even by deleting the intergration and trying to add it again leeds to an "unknown error". I simply cannot get it to work any more. I give up! I will use the MQTT interface and implement it by reading the values from MQTT. Thanks again for your work. Cheers

[umtauscher]

This is the error that I get when I try to reinstall the Openterm integration:

Logger: aiohttp.server Source: components/opentherm_gw/config_flow.py:65 First occurred: 09:41:30 (1 occurrences) Last logged: 09:41:30 Error handling request

Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request resp = await request_handler(request) File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle resp = await handler(request) File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 136, in handle result = await result File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 177, in post return await super().post(request, flow_id) File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 62, in wrapper result = await method(view, request, *args, **kwargs) File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 109, in post result = await self._flow_mgr.async_configure(flow_id, data) File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 277, in async_configure result = await self._async_handle_step( File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 359, in _async_handle_step result: FlowResult = await getattr(flow, method)(user_input) File "/usr/src/homeassistant/homeassistant/components/opentherm_gw/config_flow.py", line 68, in async_step_init res = await asyncio.wait_for(test_connection(), timeout=10) File "/usr/local/lib/python3.10/asyncio/tasks.py", line 456, in wait_for return fut.result() File "/usr/src/homeassistant/homeassistant/components/opentherm_gw/config_flow.py", line 65, in test_connection return status[gw_vars.OTGW].get(gw_vars.OTGW_ABOUT) TypeError: 'bool' object is not subscriptable

[mvn23]

That looks like #75589, the fix for which is almost ready.

As for the sensors, the documentation clearly states that not all sensors will have useful values. We are limited by the OpenTherm protocol and how the gateway is able to influence it. All we can do with that is report exactly what we see in the protocol, which may not always be useful. There is no way to tell in advance if a value will stay up to date and no way to determine afterwards whether a value is still valid or not. This is left to the user which may have to do some research and go through some trial and error to determine this. This is - in part - why all sensors and binary sensors are disabled by default.

[umtauscher]

I have a new log for you: This is after the update to 2022.8 I can see the connection on the OT-Monitor webpage, so I know the address ist correct. The connection vanishes after a few seconds.

Hope this help homeassistant.log s

[mvn23]

2022-08-04 18:50:23.936 ERROR (MainThread) [pyotgw.connection] The serial device on socket://otgw.voyager95.de:7686 is not responding. Will keep trying.

That means that you're not getting the response we expect from the gateway. For further investigation I would need a debug log of the pyotgw library as well.

[umtauscher]

hope this helps: pyotgw.log

[umtauscher]

homeassistant.log maybe this log is better..

[mvn23]

Thanks! It looks like you have the same issue as mentioned in https://github.com/home-assistant/core/issues/75781#issuecomment-1202771629 Basically, the command is being sent, but it seems the gateway is either not receiving it or not responding to it. You can check this in otmonitor by sending free format commands from Options->Miscellaneous and checking the log for a response (e.g. send PR=A and check for the firmware version in the response). In the other case there was no response. That time it was fixed by updating and power cycling the gateway as described in https://github.com/home-assistant/core/issues/75781#issuecomment-1203146707.

[umtauscher]

Thanks, powercycling of the interface did the trick. I never had to do this up to now. It was running for years. What might have caused this, was the deinstallation of the Opentherm_gw when I tried to debug the problem. (just a thought) It was totally unclear to me, that the otgw-interface could cause serial errors on the OT-Monitors network interface.

Thanks for all your help. Cheers

[umtauscher]

Thinking a little bit more about it, it seems strange, that the OT-Monitor which is using the same communication works as expected. Every temperatur was shown properly during that time. Maybe the is still a protocol bug in the Openterm_gw implementation?

[mvn23]

That's due to the fact that we require commands to be working since pyotgw 2.0.0 while otmonitor works without them as well (albeit only partially).

[umtauscher]

Thanks, what about transparent mode which I switched on in the Server on OT Monitor. Is that helpful, needed or bad?