MQTT compatibility with EMS-ESP V2

[lmdecroos]

The syntax of MQTT commands to write back to version 2 of the EMS-ESP software are changed https://github.com/proddy/EMS-ESP/tree/v2#mqtt-commands

Are you planning to update the domoticz plugin ?

[bbqkees]

I'm going to update the plugin when V2 is near release. There are continuous changes to the beta's, this command structure might change again as well.

I do not plan to integrate all new commands, because they would each need a Domoticz device. The python plugins have a limit of about 256 devices and this plugin has a lot of devices in use already. Commands like setting the building type are not important so I won't 'waste' a precious Domoticz ID on those.

[sq4bja]

Now, there is a 2.0 stabile release. It would be nice if the WWtemperature could be changed, as well as the heating and boiler modes. Now I have to do this through external scripts that execute telnet sequences.

[Yomark1]

I too have issues that are probably related. I installed a new domoticz server two days ago on Ubuntu server 20, and ordered and installed the fantastic Gateway Premium II(installed firmware 2.01). Installed the Domoticz plugin, and it added all the devices like magic. However, setting thermostat temperature does not seem to work. Using telnet on the device itself does work.

[bbqkees]

The MQTT command format has changed completely in 2.0.1. Currently the plugin sends the old format instead of the new. That's why the thermostat command does not work. I've planned an update next week.

On Tue, 29 Sep 2020, 14:20 Yomark1, notifications@github.com wrote:

I too have issues that are probably related. I installed a new domoticz server two days ago on Ubuntu server 20, and ordered and installed the fantastic Gateway Premium II(installed firmware 2.01). Installed the Domoticz plugin, and it added all the devices like magic. However, setting thermostat temperature does not seem to work. Using telnet on the device itself does work.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bbqkees/ems-esp-domoticz-plugin/issues/17#issuecomment-700665002, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACSGZAW6XAQSTAXCZF2PA63SIHGH3ANCNFSM4PTUYW2Q .

[bbqkees]

So in the dev-2 branch there is the first start of the plugin version for EMS-ESP V2.1. You can try it out. It works on my Moduline 400 thermostat.

To switch between branches see: https://github.com/bbqkees/ems-esp-domoticz-plugin/tree/dev#switching-from-master-to-dev-branch

Still a lot of work to do in the coming days so keep an eye on the changelog.

[sq4bja]

Is it necessary to remove all devices added by the dev version?

[bbqkees]

Is it necessary to remove all devices added by the dev version?

No that should not be necessary. The dev version uses the same plugin name and IDX numbers. If you replace the code or pull it in you do need to restart Domoticz.

[sq4bja]

Ok, After upgrade look OK. I have RC310 with GB072 boiler. But the same error in domoticz log:

 2020-10-06 23:38:44.510 Error: (BUDERUS) 'onMessage' failed 'NameError'.
2020-10-06 23:38:44.510 Error: (BUDERUS) ----> Line 817 in '/home/pi/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py', function onMessage
2020-10-06 23:38:44.510 Error: (BUDERUS) ----> Line 756 in '/home/pi/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py', function onMessage
2020-10-06 23:38:44.510 Error: (BUDERUS) ----> Line 113 in '/home/pi/domoticz/plugins/ems-esp-domoticz-plugin/mqtt.py', function onMessage
2020-10-06 23:38:44.510 Error: (BUDERUS) ----> Line 788 in '/home/pi/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py', function onMQTTPublish 

[bbqkees]

Yes that error is only once at startup. You can ignore that one.

[lmdecroos]

@bbqkees : thanks for updating the plug-in I have replaced the 1.1b2 version with the new 1.2b3 version, I will monitor the behaviour

On 6 Oct 2020, at 23:43, Kees notifications@github.com wrote:

Yes that error is only once at startup. You can ignore that one.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bbqkees/ems-esp-domoticz-plugin/issues/17#issuecomment-704569161, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDW44YE6AM244MWPJGEXYTSJOFQDANCNFSM4PTUYW2Q.

[lmdecroos]

I had during the day 27 times the error from the plugin

2020-10-07 19:14:15.944 Error: (EMS) 'onMessage' failed 'NameError'. 2020-10-07 19:14:15.944 Error: (EMS) ----> Line 817 in '/home/lmd/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py', function onMessage 2020-10-07 19:14:15.944 Error: (EMS) ----> Line 756 in '/home/lmd/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py', function onMessage 2020-10-07 19:14:15.944 Error: (EMS) ----> Line 113 in '/home/lmd/domoticz/plugins/ems-esp-domoticz-plugin/mqtt.py', function onMessage 2020-10-07 19:14:15.944 Error: (EMS) ----> Line 788 in '/home/lmd/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py', function onMQTTPublish

The errors happens when the boiler updates "Boiler Service code" and "Boiler Service code number” index 18 and index 19

On 7 Oct 2020, at 07:39, Luk-Marie Decroos lmd@telenet.be wrote:

@bbqkees : thanks for updating the plug-in I have replaced the 1.1b2 version with the new 1.2b3 version, I will monitor the behaviour

On 6 Oct 2020, at 23:43, Kees <notifications@github.com mailto:notifications@github.com> wrote:

Yes that error is only once at startup. You can ignore that one.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bbqkees/ems-esp-domoticz-plugin/issues/17#issuecomment-704569161, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDW44YE6AM244MWPJGEXYTSJOFQDANCNFSM4PTUYW2Q.

[lmdecroos]

I found a syntax error when updating the mode for the thermostat

The 1.2b4 is publishing :

{"cmd":"mode" ,"data":day, "hc":2}

This should be

{"cmd":"mode","data”:"day","hc”:"2"}

The number of the HC should also between quotations

An enhancement question : Would it be possible to have both “mode" and "mode type” displayed ? Mode: auto Mode type: day

On 7 Oct 2020, at 20:59, Luk-Marie Decroos lmd@telenet.be wrote:

I had during the day 27 times the error from the plugin

2020-10-07 19:14:15.944 Error: (EMS) 'onMessage' failed 'NameError'. 2020-10-07 19:14:15.944 Error: (EMS) ----> Line 817 in '/home/lmd/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py', function onMessage 2020-10-07 19:14:15.944 Error: (EMS) ----> Line 756 in '/home/lmd/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py', function onMessage 2020-10-07 19:14:15.944 Error: (EMS) ----> Line 113 in '/home/lmd/domoticz/plugins/ems-esp-domoticz-plugin/mqtt.py', function onMessage 2020-10-07 19:14:15.944 Error: (EMS) ----> Line 788 in '/home/lmd/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py', function onMQTTPublish

The errors happens when the boiler updates "Boiler Service code" and "Boiler Service code number” index 18 and index 19

On 7 Oct 2020, at 07:39, Luk-Marie Decroos <lmd@telenet.be mailto:lmd@telenet.be> wrote:

@bbqkees : thanks for updating the plug-in I have replaced the 1.1b2 version with the new 1.2b3 version, I will monitor the behaviour

On 6 Oct 2020, at 23:43, Kees <notifications@github.com mailto:notifications@github.com> wrote:

Yes that error is only once at startup. You can ignore that one.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bbqkees/ems-esp-domoticz-plugin/issues/17#issuecomment-704569161, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDW44YE6AM244MWPJGEXYTSJOFQDANCNFSM4PTUYW2Q.

[bbqkees]

@lmdecroos According to the EMS-ESP wiki both strings and numbers are allowed for hc. The data parameter should indeed be between quotations. I'll change it in the next update.

Can you elaborate on the enhancement question? Not sure what you mean.

[lmdecroos]

The thermostat mode as displayed in domoticz is

it displays only themostat mode : who is "auto" it doesn't display the mode type : who is " night"

from MQTT message : {"hc1":{"seltemp":15,"currtemp":19.2,"daytemp":19,"nighttemp":15,"holidaytemp":17,"nofrosttemp":2.5,"heatingtype":1,"targetflowtemp":5,"offsettemp":0,"designtemp":75,"summertemp":17,"mode":"auto","modetype":"night"},"hc2":{"seltemp":13.5,"currtemp":16.8,"daytemp":18,"nighttemp":13.5,"holidaytemp":17,"nofrosttemp":2.5,"heatingtype":1,"targetflowtemp":5,"offsettemp":0,"designtemp":75,"summertemp":17,"mode":"night","modetype":"night"}}

from the ems-esp terminal : Mode: auto Mode type: night

[bbqkees]

Ok which thermostat do you have? With a Moduline 400 (~RC35) I only have: {"time":"06:42:31 30/04/2000","dampedtemp":-10,"hc1":{"seltemp":18,"currtemp":19.1,"mode":"day"}}

The EMS-ESP API does not make a distinction between mode and modetype.

[lmdecroos]

I was only considering to display both mode and mode type in the domoticz GUI.

It’s correct that you are only able to change the mode and not the mode type via the API & MQTT The mode type is to my knowledge set by the program you have set on your thermostat The mode is overwriting the mode type, this is the same as you would push the auto, day or night button on your thermostat.

I have a RC35 thermostat

On 7 Oct 2020, at 22:32, Kees notifications@github.com wrote:

Ok which thermostat do you have? With a Moduline 400 (~RC35) I only have: {"time":"06:42:31 30/04/2000","dampedtemp":-10,"hc1":{"seltemp":18,"currtemp":19.1,"mode":"day"}}

The EMS-ESP API does not make a distinction between mode and modetype.

https://user-images.githubusercontent.com/10775682/95384495-fcb5a980-08ec-11eb-9b0d-abe6ea9cb4e9.png — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bbqkees/ems-esp-domoticz-plugin/issues/17#issuecomment-705177969, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDW44467JAULP3KK3ZSCNDSJTF7FANCNFSM4PTUYW2Q.

[bbqkees]

Ok I'll add the mode type as well.

[Yomark1]

I get segmentation fault 11 when enabling the DEV-2 version 1.2b4.

`2020-10-07 22:59:15.188  Error: (Nefit) 'onMessage' failed 'NameError'.
2020-10-07 22:59:15.188  Error: (Nefit) ----> Line 834 in '/home/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py'
2020-10-07 22:59:15.188  Error: (Nefit) ----> Line 773 in '/home/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py'
2020-10-07 22:59:15.188  Error: (Nefit) ----> Line 113 in '/home/domoticz/plugins/ems-esp-domoticz-plugin/mqtt.py'
2020-10-07 22:59:15.188  Error: (Nefit) ----> Line 805 in '/home/domoticz/plugins/ems-esp-domoticz-plugin/plugin.py'
2020-10-07 22:59:16.240  Error: Domoticz(pid:43112, tid:43115('PluginMgr')) received fatal signal 11 (Segmentation fault)
2020-10-07 22:59:16.240  Error: siginfo address=0xa868, address=0x7f6426a0424b
2020-10-07 22:59:16.244  Error: Failed to start gdb, will use backtrace() for printing stack frame

2020-10-07 22:59:16.272  Error: #0  /home/domoticz/domoticz :  + 0x391966 [0x55743c0c6966]
2020-10-07 22:59:16.272  Error: #1  /home/domoticz/domoticz : signal_handler(int, siginfo_t*, void*) + 0x2b5 [0x55743c0c7525]
2020-10-07 22:59:16.272  Error: #2  /lib/x86_64-linux-gnu/libpthread.so.0 :  + 0x153c0 [0x7f6426a043c0]
2020-10-07 22:59:16.272  Error: #3  /lib/x86_64-linux-gnu/libpthread.so.0 : raise + 0xcb [0x7f6426a0424b]
2020-10-07 22:59:16.272  Error: #4  /lib/x86_64-linux-gnu/libpthread.so.0 :  + 0x153c0 [0x7f6426a043c0]
2020-10-07 22:59:16.272  Error: #5  /lib/x86_64-linux-gnu/libpython3.8.so :  + 0x26b5a1 [0x7f64256ff5a1]
2020-10-07 22:59:16.272  Error: #6  /lib/x86_64-linux-gnu/libpython3.8.so : PyDict_SetItem + 0x24b [0x7f64256ffe8b]
2020-10-07 22:59:16.272  Error: #7  /lib/x86_64-linux-gnu/libpython3.8.so : PyDict_SetItemString + 0x39 [0x7f64256fc019]
2020-10-07 22:59:16.272  Error: #8  /home/domoticz/domoticz :  + 0x78a3ba [0x55743c4bf3ba]
2020-10-07 22:59:16.272  Error: #9  /home/domoticz/domoticz : Plugins::CPluginProtocolMQTT::ProcessInbound(Plugins::ReadEvent const*) + 0x8d5 [0x55743c4c2ef5]
2020-10-07 22:59:16.272  Error: #10 /home/domoticz/domoticz : Plugins::CPluginMessageBase::Process() + 0x37 [0x55743c03b2f7]
2020-10-07 22:59:16.272  Error: #11 /home/domoticz/domoticz : Plugins::CPluginSystem::Do_Work() + 0xbce [0x55743c4ba47e]
2020-10-07 22:59:16.272  Error: #12 /home/domoticz/domoticz :  + 0xaeddbf [0x55743c822dbf]
2020-10-07 22:59:16.272  Error: #13 /lib/x86_64-linux-gnu/libpthread.so.0 :  + 0x9609 [0x7f64269f8609]
2020-10-07 22:59:16.272  Error: #14 /lib/x86_64-linux-gnu/libc.so.6 : clone + 0x43 [0x7f64267a1293]
`

Switched to Domoticz Beta --> same issue Build Domoticz from source and starting from scratch(clean database), same issue.

When installing this Ubuntu 20.04 server last week, I had to switch to python3.8-dev package to get domoticz running(before that, it also crashed when installing a plugin), but I still think there is something fishy with Python and Domoticz on my(fresh) system. However, the old plugin version and other plugins did work fine eventually with python-dev. This new version does not. (I don't expect you can do anything about this, but just for reference).

[bbqkees]

So is it specifically this plugin that crashes the system or is it enabling any plugin? There is nothing really 'new' in this plugin version. Mostly moved around some stuff, added a few functions and changed the MQTT topics.

[bbqkees]

@lmdecroos I have made an update for the mode type. The service code does not give an error here. A long time ago I changed the device type to a text sensor of the those two parameters. Maybe that's the problem.

[Yomark1]

@bbqkees Tnx for your reply. No, the weird thing is, the old plugin runs fine. Other plugins too. That being said, I had these weird crashes also on other plugins when I used the default ubuntu python package, but those went away when I started using the -dev package(like someone suggested on the domoticz forum for someone else with similar problems).
Anyway, i'm not the only one with issues like this. Al lot of forum threads on the domoticz forum regarding "('PluginMgr')) received fatal signal 11 (Segmentation fault)", regarding a lot of different plugins(that are working for other people) but with no solution. Although some people did manage to fix it with compiling domoticz from source. However, that also didn't improve things in my case. Seems like domoticz is not going to work for me. It's a pity, because although it seems a bit less graphically polished then for example home assistant, it is actually far simpler(scripting DzVents ) and more powerful to use.

[bbqkees]

@Yomark1 I started my first home automation stuff with Domoticz many years ago and in the mean time I got so many devices and scripts I did not bother to migrate to Home Assistant when it became really good. I'm using Domoticz on my own Debian server for personal use and on several Raspberry Pi's for testing purposes. Sometimes you get vague errors but usually it ends up being a quirk of your particular Linux version.

[Yomark1]

@bbqkees tnx for the response. This is all wildly off-topic, but "Ubuntu" seems to be the only Linux distribution mentioned in the documentation "How to install" besides of course the manuals for the pi's.

So i just removed everything, and installed the older ubuntu server 18.04 from scratch. Followed Domoticz the manual, which does not mention much else except executing "curl -sSL install.domoticz.com | sudo bash". And that is great. Does a lot of checks for available binaries, and seems to think everything is great, and installs Domoticz. However, it isn't great, because it then crashes on startup. Why only becomes clear when enabling -log in the init script, and the problem is it can't find "libpython", while libpython is installed. The same thing was happening the first time, so fixed it by installing libpython3.8-dev (why - because I found in some forumthread from 2015 someone had a similar issue) Now it starts.

Now finally on topic....the Plugin :) It does work , no crashes. So there is something not right with the python version in Ubuntu 20.04 or with the implementation in domoticz.

Status: (Nefit) Initialized version 1.2b6 2020-10-08 12:26:36.826 (Nefit) Debug logging mask set to: PYTHON PLUGIN DEVICE 2020-10-08 12:26:36.826 Status: (Nefit) Entering work loop. 2020-10-08 12:26:36.826 (Nefit) checkDevices called 2020-10-08 12:26:36.827 (Nefit) Topiclist is: 2020-10-08 12:26:36.827 (Nefit) ems-esp/thermostat_data, ems-esp/boiler_data, ems-esp/sensor_data, ems-esp/mixing_data, ems-esp/solar_data, ems-esp/hp_data, ems-esp/heating_active, ems-esp/tapwater_active, ems-esp/status, ems-esp/info

The issue i'm having, it that it doesn't update the thermostat still. (was on 20.5, want to set it to 20.0)

2020-10-08 12:27:16.854 (Nefit) onMessage called with: PINGRESP 2020-10-08 12:27:22.565 (Nefit) Command: Set Level (20.0 2020-10-08 12:27:22.565 (Nefit) onCommand called for Unit 112: Parameter 'Set Level', Level: 20.0 2020-10-08 12:27:22.565 (Nefit) MqttClient::Publish ems-esp/thermostat ({"cmd":"temp" ,"data":20.0, "hc":1}) some time after that 020-10-08 12:27:24.087 (Nefit) onMessage called with: PUBLISH 2020-10-08 12:27:24.238 (Nefit) MQTT message: ems-esp/thermostat_data b'{"hc1":{"seltemp":20.5,"currtemp":20.5}}' 2020-10-08 12:27:24.239 (Nefit) deviceId 111 value has not changed. Device not updated 2020-10-08 12:27:24.239 (Nefit) deviceId 112 value has not changed. Device not updated 2020-10-08 12:27:24.239 (Nefit) onMessage called with: PUBLISH

I'm not sure on one thing, the MQTT format in the EMS-ESP(version 2.01) was set to home assistant by me(testing with HI), and I set it back(although i'm not 100%sure) to "Nested" (and rebooted the device). That is the correct setting, right?

Thank you for your great work.

[bbqkees]

It should be nested indeed, but that's only important for the data topics, not the command topics. Can you update EMS-ESP to V2.1? I'm using V2.1b5 and there it works. Which thermostat are you using?

[sq4bja]

In newest version of plugin and EMS-ESP do not work "EMS thermostat setpoint HC1". Setting thermostat temperature does not seem to work. Via Telnet does work. After upgrade i get new device with strange selector:

I have RC310 controller. Tom

[Yomark1]

Flashed to version V2.1b6 WORKING! :) (For people wanting to do the same, here it is: https://github.com/proddy/EMS-ESP/tree/firmware ) (For people wondering what the username/password is on the webinterface in 2.1, it's admin/admin)

Tnx Kees!!

[bbqkees]

@sq4bja that new selector is the 'mode type' that is also reported for some thermostats. Its read only.

Setting the thermostat only works in V2.1 as the API commands are not yet implemented in V2.0.1. There should be a release of 2.1 soon.

[sq4bja]

@bbqkees Ok, upgrade to 2.1.0b6 solve he thermostat problem. It works. I`m waiting for other switches and controls to be active.

[bbqkees]

@sq4bja can you be more specific about which switches and controls? Its pretty easy to add now.

[sq4bja]

It would be nice if you could set the warm water temperature, warm water mode, One time heating warm water and Heating mode (summer, winer, auto).

And now do not work Boiler warm water current temperature.

Tom

[maxverstappen]

It would be nice if you could set the warm water temperature, warm water mode, One time heating warm water and Heating mode (summer, winer, auto).

And now do not work Boiler warm water current temperature.

Tom In the meanwhile you can try this.

!/bin/sh

HOST='xx.xx.xx.xx' CMD='boiler wwtemp 50' ( echo open "$HOST" sleep 2 echo "$CMD" sleep 2 echo "exit" ) | telnet

[sq4bja]

Yes, it'm doing it that way, but MQTT is a better protocol then telnet, so i think, it should work.

[bbqkees]

@sq4bja At this moment I can only add wwtemp as the others are not in the API yet. https://emsesp.github.io/docs/#/API Boiler comfort mode and ww mode are already in the plugin.

[lmdecroos]

@bbqkees : thanks for adding the mode type the syntax for the "LevelNames" is all lowercase and should be "off|auto|day|night|manual|heat|eco|nofrost" you can off course change the names in the switches but it would be better if the syntax is already correct in the plugin

It took me a while before I discovered why the mode type switch was not updating :)

would it be possible to combine the mode and mode type in one switch (domoticz device) ?

[bbqkees]

Ok i'll change it in the next update. I did take into account using only lowercase for setting the command but did not think of the other way around.

I can't really combine mode and mode type as those are two different parameters. I also have no idea what the full set of values for both should be.

[sq4bja]

Update: (RC310) - below devices in domoticz are not updated even though the correct parameters are in ems-esp:

Boiler warm water current temperature - no updates - doesn't work (very importent value!) Boiler warm water heating - lives its own life - does not correspond to reality boiler heating working minutes - is not updated Heating temperature - is not updated

[bbqkees]

Ok I see the wWCurTmp should be wWCurTemp. Don't really see why heatWorkMin would not update. heatingTemp is not in it yet.

It also appears that heating_active and tapwater_active are moved from their own topics to boiler_data in 2.1b6 so I will have to look at that as well.

[sq4bja]

Thats right. After changing wWCurTmp to wWCurTemp temperature comes back 👍

[lmdecroos]

@bbqkees regarding the error on device 18 and 19, after updating the device in domoticz the errors are gone. So it was due a mismatch in parameter

On 8 Oct 2020, at 11:29, Kees notifications@github.com wrote:

@lmdecroos https://github.com/lmdecroos I have made an update for the mode type. The service code does not give an error here. A long time ago I changed the device type to a text sensor of the those two parameters. Maybe that's the problem.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bbqkees/ems-esp-domoticz-plugin/issues/17#issuecomment-705449938, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDW445MTC7Q6DHTNY4IVM3SJWBBHANCNFSM4PTUYW2Q.

[th3sniper86]

I'm using a Moduline 400. These are the topics that are published to MQTT. No HC1 etc anymore. My thermostat devices in domoticz are not updated if I use the latest firmware (2.1.0b6) with the dev-2 plugin.

[bbqkees]

@th3sniper86 You have to set the MQTT format to 'nested'.

[th3sniper86]

Thanks @bbqkees. problem solved.

[bbqkees]

I just made a quick typo fix for wWCurTemp. Its the only change in 1.2b7. After the weekend I'll continue on the plugin.

[bbqkees]

I pushed all current changes to main.

'dev-2' is now 'dev' again, so you need to change the branch as well.

[bbqkees]

I'm closing this one as the plugin is ready for V2.1.