search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
70.46k stars 29.39k forks source link

Opentherm Gateway component not working #17263

Closed Wummeke closed 5 years ago

Wummeke commented 5 years ago

Home Assistant release with the issue: 0.79.3

Last working Home Assistant release (if known): None

Operating environment (Hass.io/Docker/Windows/etc.): I am running Home Assistant from Docker

Component/platform: https://www.home-assistant.io/components/climate.opentherm_gw/

Description of problem: Component does not work basicly. I sometimes have to reboot Home Assistant a few times to get the component going:

2018-10-08 22:23:24 WARNING (MainThread) [homeassistant.components.climate] Setup of platform opentherm_gw is taking over 10 seconds.
2018-10-08 22:23:30 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/helpers/entity_platform.py", line 345, in _async_add_entity
    await entity.async_added_to_hass()
  File "/usr/src/app/homeassistant/components/climate/opentherm_gw.py", line 66, in async_added_to_hass
    await self.gateway.connect(self.hass.loop, self._device)
  File "/usr/local/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 55, in connect
    await self.get_reports()
  File "/usr/local/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 170, in get_reports
    OTGW_DEFAULT_TIMEOUT, loop=self.loop)
  File "/usr/local/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

and When it finally works, it stops working after a few minutes. When I try to change the temperature an error is added to the log:

2018-10-08 22:13:51 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/helpers/service.py", line 226, in _handle_service_platform_call
    await func(entity, data)
  File "/usr/src/app/homeassistant/components/climate/__init__.py", line 679, in async_service_temperature_set
    await entity.async_set_temperature(**kwargs)
  File "/usr/src/app/homeassistant/components/climate/opentherm_gw.py", line 174, in async_set_temperature
    temp)
  File "/usr/local/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 91, in set_target_temp
    self._protocol.issue_cmd(cmd, value), timeout, loop=self.loop))
  File "/usr/local/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

My openthermgateway is connected to a nodeMCU with ESP Easy running ser2net.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

climate:
  - platform: opentherm_gw
    device: socket://192.168.0.6:6638
    name: Thermostat
    precision: 0.5

Additional information: Here a log of a 'successful' connection after reboot and then almost imediately a crash of the component when I try to set a temperature:

2018-10-08 22:29:12 INFO (MainThread) [homeassistant.components.climate] Setting up climate.opentherm_gw
2018-10-08 22:29:17 DEBUG (MainThread) [homeassistant.components.climate.opentherm_gw] Connected to Thermostat on socket://192.168.0.6:6638
2018-10-08 22:29:17 DEBUG (MainThread) [homeassistant.components.climate.opentherm_gw] Received report: {'control_setpoint': 10.0, 'otgw_about': 'OpenTherm Gateway 4.2.5', 'otgw_build': '17:59 20-10-2015', 'otgw_clockmhz': '4 MHz', 'otgw_gpio_a': 0, 'otgw_gpio_b': 0, 'otgw_gpio_a_state': 0, 'otgw_gpio_b_state': 0, 'otgw_led_a': 'F', 'otgw_led_b': 'X', 'otgw_led_c': 'O', 'otgw_led_d': 'T', 'otgw_led_e': 'P', 'otgw_led_f': 'C', 'otgw_mode': 'G', 'otgw_setpoint_ovrd_mode': 'N', 'otgw_smart_pwr': 'Low power', 'otgw_thermostat_detect': 'D', 'otgw_setback_temp': '16.00', 'otgw_ignore_transitions': 1, 'otgw_ovrd_high_byte': 1, 'otgw_vref': 3, 'otgw_dhw_ovrd': 'A', 'master_ch_enabled': 0, 'master_dhw_enabled': 1, 'master_cooling_enabled': 0, 'master_otc_enabled': 0, 'master_ch2_enabled': 0, 'slave_fault_indication': 0, 'slave_ch_active': 0, 'slave_dhw_active': 0, 'slave_flame_on': 0, 'slave_cooling_active': 0, 'slave_ch2_active': 0, 'slave_diagnostic_indication': 0, 'remote_transfer_dhw': 0, 'remote_transfer_max_ch': 0, 'remote_rw_dhw': 0, 'remote_rw_max_ch': 0, 'slave_max_relative_modulation': 0.0, 'slave_max_capacity': 0, 'slave_min_mod_level': 0, 'room_setpoint': 0.0, 'relative_mod_level': 0.0, 'ch_water_pressure': 0.0, 'room_temp': 0.0, 'ch_water_temp': 0.0, 'dhw_temp': 0.0, 'outside_temp': 0.0, 'return_water_temp': 0.0, 'slave_dhw_max_setp': 0, 'slave_dhw_min_setp': 0, 'slave_ch_max_setp': 0, 'slave_ch_min_setp': 0, 'dhw_setpoint': 0.0, 'max_ch_setpoint': 90.0, 'ch_burner_starts': 0, 'ch_pump_starts': 0, 'dhw_pump_starts': 0, 'dhw_burner_starts': 0, 'ch_burner_hours': 0, 'ch_pump_hours': 0, 'dhw_pump_hours': 0, 'dhw_burner_hours': 0, 'date': datetime.datetime(2018, 10, 8, 22, 29, 17, 661521)}
2018-10-08 22:30:01 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/helpers/service.py", line 226, in _handle_service_platform_call
    await func(entity, data)
  File "/usr/src/app/homeassistant/components/climate/__init__.py", line 679, in async_service_temperature_set
    await entity.async_set_temperature(**kwargs)
  File "/usr/src/app/homeassistant/components/climate/opentherm_gw.py", line 174, in async_set_temperature
    temp)
  File "/usr/local/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 91, in set_target_temp
    self._protocol.issue_cmd(cmd, value), timeout, loop=self.loop))
  File "/usr/local/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
MartinHjelmare commented 5 years ago

CC @mvn23

mvn23 commented 5 years ago

This looks like a connection issue to me. A few things you can try to confirm/troubleshoot this:

In the first scenario, you should see a few lines of 9 characters each appearing roughly every second. If not, try typing PS=0 and hit enter. If the lines start with an E there is an issue with the gateway device or config and you may want to try the troubleshooting from the OpenTherm Gateway website. If the lines don't have 9 characters each or appear garbled, there may be an issue with the ser2net config. Please configure ser2net to comply with the serial port specifications from the OpenTherm Gateway website. This is what I use in ser2net.conf for local testing on port 8802:

127.0.0.1,8802:raw:0:/dev/otgw:9600 NONE 1STOPBIT 8DATABITS XONXOFF LOCAL -RTSCTS

In the second scenario, if the thermostat and boiler are not connected, you should see the line E00000000 appear every second IIRC. If not, try the PS=0 or PS=1 commands and post the output.

Wummeke commented 5 years ago

Seems like I have an issue in the combination of Home Assistant, my NodeMCU/OTGW and my Google Home mini. I just accidentally discovered that when I have my Google Home mini unplugged and reboot Home Assistant, the connection between HA and the NodeMCU/OTGW stays OK, until I plug my GHM back in. Not sure why this is happening and I am a bit puzzled, because OTmonitor (which also runs from a Docker container on the same host as Home Assistant) doesn't have this issue, it stays connected with the NodeMCU/OTGW (although it is now turned off, because I first that that was blocking HA from connecting to the NodeMCU/OTGW).

Wummeke commented 5 years ago

I had IP reservations on the DHCP server for all the mentioned devices. I have removed the DHCP address reservation for the Google Home Mini on my router and it seems that it solved the issue (for now).

But I have another question: is it possible that the OTGW component reconnects itself after it got disconnected? Or let it at least attempt to reconnect? Now I have to restart HA completely to achieve that.

mvn23 commented 5 years ago

Automatic reconnection may be added in a future release, but it is quite low on my to-do list. In it's normal config, the gateway is a serial device that may be connected through a serial-to-USB interface, so any disconnect or intermittent connection would be considered an abnormality. Using ser2net is a way to make the setup more versatile, but as you noticed it is definitely not perfect. For a more stable connection I can only recommend to use a wired network connection rather than wifi. In my production setup I have the gateway connected to a DreamPlug which is forwarding the serial connection over gigabit wired ethernet to my Home Assistant instance. This has been running stable for weeks now.

Wummeke commented 5 years ago

@mvn23 Hopefully you can move it up on your to-do-list a little someday. I don't have the possibility to connect the OTGW wired to my Home Assistant instance unfortunately. Thanks for the time and effort you've put in this component. Too bad I can't use at this moment.

infernix commented 5 years ago

I'll chime in here - I have an OTGW with an ESP8266 using a software serial build I made and have otmonitor talking to it for more than a year. Reasoning for that setup is in the repo description.

I had to build a wrapper script around otmonitor because I suspect otmonitor isn't queueing commands and sometimes HASS will send a command to otmonitor while it's already sending a command to the OTGW and that freezes/breaks otmonitor. So I've also scripted a restart of otmonitor in case it isn't replying to commands that are being sent.

As for the usage in HASS, I don't have a physical thermostat so my OTGW is in "standalone" mode. I send CH=1 if my HASS rules determine that there's a need to heat the house/rooms, and CH=0 when heat demand has been met. I also set the setpoint CS, modulation MM and occasionally hot water setpoint SW. I've also set up a binary sensor that sends periodic PR=A requests to determine whether if the OTGW connection is still alive.

I'd gladly dump otmonitor, but since the OTGW is really a critical component, controlling it has to be reliable and fault-tolerant. Having the heater on for too long has really bad effects :)

mvn23 commented 5 years ago

@WumsWatchFace this may be related, Google Home devices seem to be affected as well. However, firmware should have been updated automatically already.

@infernix your setup will not be officially supported as you're looking to replace your thermostat rather than control it. However, when I start working on the implementation of services I will take it into consideration while deciding which commands to implement.

thefoxmangit commented 5 years ago

I am also running my OTGW with a NodeMCU and am trying to connect it to Home Assistant. From my windows machine, OTmonitor works perfectly. I can ping the NodeMCU from the Hass.io docker instance. However, the component does not work. I have no Google home appliances running.

In the log the following is mentioned:

2018-10-16 22:17:04 WARNING (MainThread) [homeassistant.components.climate] Setup of platform opentherm_gw is taking over 10 seconds.
2018-10-16 22:17:06 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 352, in _async_add_entity
    await entity.async_added_to_hass()
  File "/usr/local/lib/python3.6/site-packages/homeassistant/components/climate/opentherm_gw.py", line 66, in async_added_to_hass
    await self.gateway.connect(self.hass.loop, self._device)
  File "/usr/local/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 55, in connect
    await self.get_reports()
  File "/usr/local/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 170, in get_reports
    OTGW_DEFAULT_TIMEOUT, loop=self.loop)
  File "/usr/local/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
mvn23 commented 5 years ago

@fmslottje I have set up a docker container with hass.io just now for testing and I can't reproduce your issue so far. It did give me the opportunity to look around in the hass.io environment a bit. Try to connect to your gateway from the command line within the docker container using nc <host> <port>. If that produces output, then please post a part (~20-30 lines) of the output you get. If you don't get any output, even after resetting the gateway, then it is definitely a connection issue (firewall/nat/routing/etc.)

magin1 commented 5 years ago

@mvn23 Hi, little bit off-topic to the problem but i am looking for a tutorial to what to buy and how to set it up. i want to control my heating with Home Assistant. Could not find it anywhere else and you seem very knowledgeable about this:)

mvn23 commented 5 years ago

@magin1 please direct your question through the Proper Support Channels™

thefoxmangit commented 5 years ago

Thanks @mvn23, I have installed the SSH addon and were able to nc, part of the output is as follows (truncated):

B407B0910 A701A0000 T80190000 BC01930B3 T10010A00 BD0010A00 T001B0000 R00740000 BC074FFFF AF01B0000 T000F0000 R80750000 B4075FFFF AF00F0000 T00120000 R80760000 B4076FFFF AF0120000 T80000200 B40000200 T80190000 BC01930B3 T10010A00 BD0010A00 T00110000 BC0110000 T001B0000 R00770000 BC077FFFF AF01B0000 T80190000 BC01930B3 T10010A00 BD0010A00 T80000200 B40000200

mvn23 commented 5 years ago

Is that the actual format or does it print each message on its own line? You can use code blocks to preserve formatting by putting a line with three backticks before and after a block of text.

thefoxmangit commented 5 years ago

sorry, each message on its own line, like this:

T90394B00
B50394B00
T80190000
B4019389E
T10010A00
BD0010A00
T100E0000
BD00E0000
T00090000
R00770000
BC077FFFF
AC0090000
T80640000
R00780000
BC0785268
A40640000
T80000200
B40000200
T80190000
B40193880
T10010A00
BD0010A00
T00110000
BC0110000
T10181407
BD0181407
T80190000
B40193880
T10010A00
BD0010A00
T90100E00
B50100E00
T9014B417
R80790000
B407958C8
A5014B417
T801C0000
B401C37B3
T80000200
B40000200
T80190000
B40193880
mvn23 commented 5 years ago

Ok, the connection seems to be fine, so the component 'should' work... If I write something up for you to put in custom_components and try with that, could you then provide me a debug log?

thefoxmangit commented 5 years ago

Sure, many thanks for following this up, greatly appreciated!

mvn23 commented 5 years ago

@fmslottje please put the contents of this repo in your custom_components folder and post a debug log to Hastebin. What this does is override the opentherm_gw climate platform in the Home Assistant release with a custom version which loads a more verbose version of the library that connects to the gateway. It is made for Home Assistant v0.80.x, but should work on earlier versions as well.

silvanverschuur commented 5 years ago

I've got a similar issue. The opentherm_gw component cannot connect to my OTGW using a TCP socket. I get the following error:

2018-10-27 09:38:51 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved Traceback (most recent call last): File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/opentherm_gw.py", line 75, in connect_and_subscribe await gateway.connect(hass.loop, device_path) File "/srv/homeassistant/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 52, in connect await self.get_reports() File "/srv/homeassistant/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 165, in get_reports ret = await self._wait_for_cmd(cmd, value) File "/srv/homeassistant/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 690, in _wait_for_cmd loop=self.loop) File "/usr/lib/python3.6/asyncio/tasks.py", line 362, in wait_for raise futures.TimeoutError() concurrent.futures._base.TimeoutError

To me this looks like a connection issue. The OTGW uses an ESP module loaded with ESP link. This is working for more than a year now. I'm using a python tool that connects to the ESP and publishes values to a MQTT topic. So I know the OTGW should be accessible using python (socket).

I would love to use the opentherm_gw component to get rid of third party tools. Any help would be appreciated.

thefoxmangit commented 5 years ago

@mvn23 I have some mixed success now, sometimes it works, sometimes it doesn't, so it's kind of hard to put a finger on the problem.

I found this in the log at a moment in which the climate component didn't load:

2018-10-21 16:11:00 WARNING (MainThread) [homeassistant.loader] You are using a custom component for climate.opentherm_gw which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you do experience issues with Home Assistant.
2018-10-21 16:11:10 WARNING (MainThread) [homeassistant.components.climate] Setup of platform opentherm_gw is taking over 10 seconds.
2018-10-21 16:11:12 ERROR (MainThread) [custom_components.climate.pyotgw.pyotgw] Timed out waiting for command: PR, value: I. Are you connecting to the OpenTherm Gateway?
2018-10-21 16:11:22 ERROR (MainThread) [custom_components.climate.pyotgw.pyotgw] Timed out waiting for command: PR, value: L. Are you connecting to the OpenTherm Gateway?
2018-10-21 16:11:23 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 352, in _async_add_entity
    await entity.async_added_to_hass()
  File "/config/custom_components/climate/opentherm_gw.py", line 66, in async_added_to_hass
    await self.gateway.connect(self.hass.loop, self._device)
  File "/config/custom_components/climate/pyotgw/pyotgw.py", line 57, in connect
    await self.get_reports()
  File "/config/custom_components/climate/pyotgw/pyotgw.py", line 181, in get_reports
    OTGW_GPIO_A_STATE: int(reports[OTGW_REPORT_GPIO_STATES][0]),
KeyError: 'I'
2018-10-21 16:11:33 WARNING (SyncWorker_16) [netdisco.ssdp] Error fetching description at 192.168.1.102
2018-10-21 16:12:24 ERROR (MainThread) [homeassistant.core] Timer got out of sync. Resetting

btw, my OTGW is on 192.168.1.109 so the error on 192.168.1.102 seems unrelated

mvn23 commented 5 years ago

Please post a full log with log level set to debug. Run hass --debug to enable debug logging globally or put the following in configuration.yaml to debug only the climate component:

logger:
  logs:
    homeassistant.components.climate: debug

This will be a long log, so please use hastebin or similar.

@silvanverschuur looks like a connection issue indeed. So far I am unable to reproduce any of these issues so it's hard to trace the real problem. Debug logs are welcome though ;)

silvanverschuur commented 5 years ago

@mvn23 I've enabled DEBUG logging for the component. This is the log:

2018-10-27 13:22:21 DEBUG (MainThread) [homeassistant.components.climate.opentherm_gw] Added device Thermostat
2018-10-27 13:22:31 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/opentherm_gw.py", line 75, in connect_and_subscribe
    await gateway.connect(hass.loop, device_path)
  File "/srv/homeassistant/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 52, in connect
    await self.get_reports()
  File "/srv/homeassistant/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 165, in get_reports
    ret = await self._wait_for_cmd(cmd, value)
  File "/srv/homeassistant/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 690, in _wait_for_cmd
    loop=self.loop)
  File "/usr/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

After this stacktrace nothing is logged anymore. I've also started hass with default logging set to debug but the results were the same (besides a lot of debug information from other components).

So it looks to me that the component completely stops when the OTGW connection cannot be established.

mvn23 commented 5 years ago

Sorry, I should have specified. Debug logs with the custom_component from here.

silvanverschuur commented 5 years ago

@mvn23 The debug log with the custom component adds one stack trace to the log:

2018-10-27 13:42:08 ERROR (MainThread) [homeassistant.components.climate] Error while setting up platform opentherm_gw
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 128, in _async_setup_platform
    SLOW_SETUP_MAX_WAIT, loop=hass.loop)
  File "/usr/lib/python3.6/asyncio/tasks.py", line 358, in wait_for
    return fut.result()
  File "/home/homeassistant/.homeassistant/custom_components/climate/opentherm_gw.py", line 40, in async_setup_platform
    gateway = OpenThermGateway(config)
  File "/home/homeassistant/.homeassistant/custom_components/climate/opentherm_gw.py", line 52, in __init__
    self._device = config[CONF_DEVICE]
KeyError: 'device'

2018-10-27 13:42:18 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/opentherm_gw.py", line 75, in connect_and_subscribe
    await gateway.connect(hass.loop, device_path)
  File "/srv/homeassistant/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 52, in connect
    await self.get_reports()
  File "/srv/homeassistant/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 165, in get_reports
    ret = await self._wait_for_cmd(cmd, value)
  File "/srv/homeassistant/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 690, in _wait_for_cmd
    loop=self.loop)
  File "/usr/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
mvn23 commented 5 years ago

I assume you're running 0.81 here? The custom_component was made for 0.80. I will make one for 0.81 later on. Also, please use code formatting for log dumps by putting the text between lines with three backticks, or even better, use hastebin.

silvanverschuur commented 5 years ago

Yes, I'm using 0.81. I'll wait for the 0.81 component.

remkokruize commented 5 years ago

I have the same issue 

2018-10-29 10:40:20 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/components/opentherm_gw.py", line 75, in connect_and_subscribe
    await gateway.connect(hass.loop, device_path)
  File "/usr/local/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 52, in connect
    await self.get_reports()
  File "/usr/local/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 165, in get_reports
    ret = await self._wait_for_cmd(cmd, value)
  File "/usr/local/lib/python3.6/site-packages/pyotgw/pyotgw.py", line 690, in _wait_for_cmd
    loop=self.loop)
  File "/usr/local/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

Using this configuration:

logger:
  default: debug
  logs:
    homeassistant.components.opentherm_gw: info
    homeassistant.components.climate: info

opentherm_gw:
  device: socket://192.168.0.159:6638
  climate:
    name: Thermostat
    precision: 0.5
    floor_temperature: False
  monitored_variables:
    - room_setpoint
    - room_temp
    - room_setpoint_ovrd
    - return_water_temp
    - otgw_about

The Opentherm gateway is connected using a nodemcu running ESPEasy which uses ser2net. It does work correctly with the otmonitor. I'm running Home Assistant 0.81.1 using Docker on a Synology DS218+

2xdehelft commented 5 years ago

Yeah exact the same problem here... Same errors, hassio 0.81, OTMonitor.exe does work on ESPEasy with Ser2net....

thefoxmangit commented 5 years ago

@mvn23 i just run the debug log with custom component on 0.80 (refrained from updating for now). Hastebin log: https://hastebin.com/huqayebawu.coffeescript

I have also added another wifi access point to my setup (practically next to the OTGW) to make 100% sure that the wifi signal is ok, and at -45db RSSI that should be absolutely fine.

Many thanks for looking into this, much appreciated!

mvn23 commented 5 years ago

Thanks for the debug log. It is giving me some more insight into what's going on. At a certain point, the gateway stops responding to commands for ~50 seconds, but it keeps on sending reports during this time. Recent versions of the library skip over these unanswered commands with just a log entry, but never recover the missing values. Without a test setup in my dev environment, it will be nearly impossible to track down the exact issue here as there's a whole chain of tools at play (network, serial, etc). One thing I can do about this is add a retry for the failed commands - possibly with a delay - to obtain the missing values, but that feels rather hacky to me.

I'll order a NodeMCU soon to play around a bit, see what I can find.

thefoxmangit commented 5 years ago

Many thanks @mvn23, if I can help out in any way please let me know

mvn23 commented 5 years ago

Ok, I've been playing with a Lolin new NodeMCU v3 with ESPeasy v120 and so far it looks like it's dropping messages. I have logged the wifi data going to the device and the serial data coming out of it and there are some discrepancies. I'm not an expert on the NodeMCU and its firmware, so I can't suggest any solutions here. They may need to change something to their serial port settings (which is also used for log output btw, make sure you disable that as well with Tools>Advanced>Serial Log Level = 0) but I can't tell for sure. The logs provided by @pascalbuijs in #16739 suggest that the same issue is also present with otmonitor, but their implementation seems to ignore the missing responses. I suggest filing a bug with NodeMCU/ESPeasy as they are far more likely to find a solution for this than I am.

joep2000 commented 5 years ago

Hi @mvn23, i'm glad to see that the OTGW is now included in HA! Although I have some questions to the logic of its behaviour (setting temp override and returning to normal operation of the thermostat, various sensors not working for me), my connection is pretty stable over wifi. My setup: remeha sense thermostat and heater, wemos d1 mini with own concoction in Arduino, serving both MQTT and a telnet connection over Wifi. If of interest I can share the code.

Indeed, if the connection is lost, I need to restart the HA. However, I don't rely on HA for my heating...

mvn23 commented 5 years ago

Good to see that it's working on your setup. Which wifi chip are you using with your Arduino? (edit: nvm, just saw it's the wemos d1 mini...) I may implement an option to allow reconnecting to the gateway, either automatically on detection of a connection problem or manually through a service, as setups over a network connection seem to be used quite frequently.

For the logic behind the override and returning to manual/program operation afterwards, please refer to the documentation on the OpenTherm Gateway website, especially the treatment of Data ID 9 and the commands section (the component sends TT commands for temperature setpoint overrides). Missing sensor values are explained in the OpenTherm Gateway Hub documentation. Both topics are unrelated to this issue, please direct further support requests through the proper support channels

mvn23 commented 5 years ago

OK, for everyone seeing TimeoutErrors with the ESP8266 chips: please post the exact model you are using (board/chip version) and the firmware type and version you are running on it. ESPEasy is claiming OpenTherm Gateway support from version R124 onwards, so the test I ran earlier on R120 is moot.

joep2000 commented 5 years ago

-running Wemos D1 mini ESP8266 chip with my own firmware: OTGW_wifi.zip -running home assistant via docker. v0.81.6

on a sidenote wrt ID 9.

thefoxmangit commented 5 years ago

I am running the Nodo-shop OTGW with a NodeMCU v3, ESP8266, with ESP Easy mega-20181030

remkokruize commented 5 years ago

I have the Nodo-shop OTGW with a NodeMCU v2(ESP8266MOD) running ESP easy v147 NodeMCU v2

gdschut commented 5 years ago

Hi All, I am also trying to set up my HA with the OTGW, and I use exactly this config: https://balk.tweakblogs.net/blog/13147/opentherm-gateway-via-wifi-naar-domoticz OTGW (slightly modded) -> ESP12 (with esp-link)

If I set up telnet to the ESP I get a list of codes which look ok:

telnet 192.168.1.49
Trying 192.168.1.49...
Connected to 192.168.1.49.
Escape character is '^]'.
T80190000
BC0193A9E
T901814D1
B501814D1
T80000200
B40000200
T10011200
BD0011200
T00110000
BC0110000
T80190000
BC0193A9E
T00090000
BC0090000
T80000200
B40000200
T10011200

In the log I get this:

Nov 09 13:30:25 pi hass[5811]: 2018-11-09 13:30:25 INFO (MainThread) [homeassistant.components.climate] Setting up climate.opentherm_gw
Nov 09 13:30:25 pi hass[5811]: 2018-11-09 13:30:25 DEBUG (MainThread) [homeassistant.components.climate.opentherm_gw] Added device Thermostat
Nov 09 13:30:34 pi hass[5811]: 2018-11-09 13:30:34 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: C. Are you connecting to the OpenTherm Gateway?
Nov 09 13:30:44 pi hass[5811]: 2018-11-09 13:30:44 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: S. Are you connecting to the OpenTherm Gateway?
Nov 09 13:30:54 pi hass[5811]: 2018-11-09 13:30:54 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: A. Are you connecting to the OpenTherm Gateway?
Nov 09 13:31:04 pi hass[5811]: 2018-11-09 13:31:04 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: T. Are you connecting to the OpenTherm Gateway?
Nov 09 13:31:14 pi hass[5811]: 2018-11-09 13:31:14 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: V. Are you connecting to the OpenTherm Gateway?
Nov 09 13:31:24 pi hass[5811]: 2018-11-09 13:31:24 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: M. Are you connecting to the OpenTherm Gateway?
Nov 09 13:31:34 pi hass[5811]: 2018-11-09 13:31:34 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: P. Are you connecting to the OpenTherm Gateway?
Nov 09 13:31:44 pi hass[5811]: 2018-11-09 13:31:44 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: R. Are you connecting to the OpenTherm Gateway?
Nov 09 13:31:54 pi hass[5811]: 2018-11-09 13:31:54 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: I. Are you connecting to the OpenTherm Gateway?
Nov 09 13:32:04 pi hass[5811]: 2018-11-09 13:32:04 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: L. Are you connecting to the OpenTherm Gateway?
Nov 09 13:32:14 pi hass[5811]: 2018-11-09 13:32:14 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: B. Are you connecting to the OpenTherm Gateway?
Nov 09 13:32:24 pi hass[5811]: 2018-11-09 13:32:24 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: G. Are you connecting to the OpenTherm Gateway?
Nov 09 13:32:34 pi hass[5811]: 2018-11-09 13:32:34 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: W. Are you connecting to the OpenTherm Gateway?
Nov 09 13:32:44 pi hass[5811]: 2018-11-09 13:32:44 ERROR (MainThread) [pyotgw.pyotgw] Timed out waiting for command: PR, value: O. Are you connecting to the OpenTherm Gateway?
Nov 09 13:32:44 pi hass[5811]: 2018-11-09 13:32:44 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Nov 09 13:32:44 pi hass[5811]: Traceback (most recent call last):
Nov 09 13:32:44 pi hass[5811]:   File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
Nov 09 13:32:44 pi hass[5811]:     result = coro.send(None)
Nov 09 13:32:44 pi hass[5811]:   File "/home/pi/opt/homeassistant/lib/python3.5/site-packages/homeassistant/components/opentherm_gw/__init__.py", line 136, in connect_and_subscribe
Nov 09 13:32:44 pi hass[5811]:     await gateway.connect(hass.loop, device_path)
Nov 09 13:32:44 pi hass[5811]:   File "/home/pi/opt/homeassistant/lib/python3.5/site-packages/pyotgw/pyotgw.py", line 60, in connect
Nov 09 13:32:44 pi hass[5811]:     await self.get_reports()
Nov 09 13:32:44 pi hass[5811]:   File "/home/pi/opt/homeassistant/lib/python3.5/site-packages/pyotgw/pyotgw.py", line 196, in get_reports
Nov 09 13:32:44 pi hass[5811]:     ovrd_mode = str.upper(reports[OTGW_REPORT_SETPOINT_OVRD][0])
Nov 09 13:32:44 pi hass[5811]: TypeError: 'NoneType' object is not subscriptable

My config looks like:

opentherm_gw:
#  device: socket://ESP-12-OTGW:23
  device: socket://192.168.1.49:23
  climate:
    name: Thermostat
    precision: 0.1
    floor_temperature: False
  monitored_variables:
    - room_setpoint
    - room_temp
    - otgw_about
    - ch_water_temp
    - otgw_mode

I tried several options for the connectstring, all with the same troubles.

Any help would be appreciated! If I can be of any help in debugging or testing I would love to help! Regards, Gerben

mvn23 commented 5 years ago

Today I ran some more tests with ESP Easy mega-20181109 and unfortunately I can not reproduce the problems mentioned here. It's actually working fine with the following chain between Home Assistant and the OpenTherm Gateway:

Laptop/HA <-Wi-Fi-> NodeMCU <-Serial-USB-> Laptop/socat <-SSH tunnel-> Dreamplug/ser2net <-USB TTL-> OpenTherm Gateway

However, I did notice that I had to reboot the NodeMCU after every config change before it started working again. I used the following config under the devices tab: screenshot_2018-11-11_17-41-27 Without restarting the NodeMCU after a config change I saw the same behavior @gdschut is seeing, because the NodeMCU would stop forwarding altogether. I will try with esp-link next to see if I can reproduce the issues there.

EDIT: Can't reproduce it with esp-link either. Once I receive the 3.3v power supply I ordered I will test with an ESP01 as well.

gdschut commented 5 years ago

That is interesting. Perhaps my HA config is not ok? What is in your config file regarding the OTGW? What do you mean with "devices tab", where can I find it? Yesterday I downloaded the OTMonitor from the OTGW website, and I can succesfully run it, so I think my chain from OTGW via ESP to WIFI is ok, as also proven by my telnet output. That is why I think my HA config to connect to OTGW by hostame or IP is wrong perhaps... If you have any thoughts on this I would like to test it. Gerben

mvn23 commented 5 years ago

"Devices tab" is a part of the ESP Easy UI for configuring the serial connection (where I took the screenshot). On esp-link (v2.2.3) I only had to change the baud rate for the serial connection to make it work. I have the following in my configuration.yaml for testing:

opentherm_gw:
  device: socket://192.168.23.59:8810
  climate:
    name: Thermostat
    precision: 0.5

With esp-link the port was changed to 23.

At this point I really don't have a clue why this is happening. On older ESP Easy firmware versions I have seen some discrepancies between the data being transmitted over Wi-Fi and the data coming out of the serial connection, but on the latest version ESP_Easy_mega-20181109_normal_ESP8266_4096.bin and on esp-link v2.2.3 it all works as expected after configuring and restarting the NodeMCU. If anybody has the option to compare Wi-Fi data to serial data and finds no differences but is still experiencing issues I might be able to do something about it, but otherwise I will need to be able to reproduce the issue first.

gdschut commented 5 years ago

I doubt if it has anything to do with pyotgw, rather than with the HA configuration. Is there any possibility to get or find more logging to find out what causes the problem with the connection? Some more info on my hardware: I run HA in VENV on RaspberryPi (2B) with a lot of other stuff. My Boiler is a modern Nefit, my thermostat a Remeha Isense. My RPi is wired on ethernet, where my OTGW connects via ESP-12 on WiFi. Both RPI as HA are uptodate. I only get complaints my PIP is outdated, but even as I update I still get complaints about using an old PIP VEnv. (But I wonder if that can cause my troubles...)

andriej commented 5 years ago

My previous comment: I use OTGW from NODO-shop with NodeMCU interface. On ESPEasy R120 - otgw is working, but not with HASS (any other controller/MQTT - I can connect/use it) While trying latest esp-link or latest stable esp-link or latest ESPEasy MEGA - all of them made OTGW to restart, making both: gateway and nodemcu unusable/unstable.

current status: after updating to latest ESP Mega and configuring it properly, OTGW-hass seem to be connected. will be testing.

edit: Timed out waiting for command: TT, value: 24.5. Are you connecting to the OpenTherm Gateway? it seems that commands don't get to OTGW anyway.

also I can't see anything more really, cause my iSense thermostat allows only TC to be changed - any possibility to add this as option?

edit2: tried to change outside temp (I don't have sensor on boiler, but can show it on thermostat): Timed out waiting for command: OT, value: 8.0. Are you connecting to the OpenTherm Gateway?

normally I use (and it's working all the time): https://github.com/martenjacobs/py-otgw-mqtt via same TCP and ESPEasy

maybe there's difference the way it connects to remote hosts? pyserial is faulty or sth?

I run hass in VENV, python 3.6.6, ARM machnie

gdschut commented 5 years ago

After several restarts of HA and reboot of my combined OTGW/ESP-12 unit now it works and communication between HA and OTGW is ok. I do not understand what changed, but it works.

andriej commented 5 years ago

I use otgw with nodemcu stacked on top. Several restarts didn't change anything. HASS can read values, changing them doesn't work (error as comment above)

joep2000 commented 5 years ago

My OTGW also crashed sometimes due to some gibberish that the esp8266 sends at boot. Perhaps this is also happening at your setup? ref: https://github.com/esp8266/Arduino/issues/3047

andriej commented 5 years ago

@joep2000 the gibberish may not be the cause, as the restart of otgw is every like 10 seconds, much after it shows. Also it doesn't happen with espeasy - since R120 till latest MEGA.

mvn23 commented 5 years ago

my iSense thermostat allows only TC to be changed - any possibility to add this as option?

Can you file a separate issue with the steps you've taken to confirm that TT isn't working but TC is? Support for TC commands is already present in the library so it should not be difficult to implement, but I don't want to add another option without some more background info on the root cause of the issue.

andriej commented 5 years ago

my iSense thermostat allows only TC to be changed - any possibility to add this as option?

Can you file a separate issue with the steps you've taken to confirm that TT isn't working but TC is? Support for TC commands is already present in the library so it should not be difficult to implement, but I don't want to add another option without some more background info on the root cause of the issue.

It was quite time ago when I was testing and it's something connected to thermostat issues only, so let's for now just leave this. I will test it once again with your implementation in HASS, maybe things have changed.

For now the only thing I'm missing is oppoturnity to get it working with current setup. :-) just wondering, maybe it's somehow fault of pyserial? line endings? (CR; LF etc.?), coding?

this is how OTGW+NodeMCU looks like - I don't really have control of which pins are used or unused for communication with OTGW

also, what was needed to be set on ESP side to get it working (otherwise no communication) was "WD I2C Address:" for value 38 (decimal) Didn't see that option in esp-link tho