Open proddy opened 9 months ago
Oh, the heating timer for RC300 are 0x1C3-ff, and 0x349-ff. with mode instead of temperature setting. We have to split RC300 - RC310 - BC400. But product-id 158 includes RC300 and RC310, maybe second identifier (unknown).
But as mentioned in the lot of discussions before, i don't know how we get that much data in ram and mqtt. Each program has 42 entries with Day-of-week, time and mode/temperature for progA, progB, DHW, circulation: 504 entities to display, holiday modes are in similar range. For RC35 we use only a single switchpoint to show/edit with a command to change the index, but this way was always not accepted by the users.
but this way was always not accepted by the users.
yes, I can understand that argument. Does it still make sense to raise the RC310 to the same level as the RC35 or leave it as it is?
For the holiday modes check #275. Same problem with a lot of entities.
The RC35 is dfferent, it's also 42 entries, but each has day, time, mode stored, We can set all 42 to monday, or 1.entry to Tu, second to monday, 3rd not set, 4th to mo, etc. The RC3x0 have groups with 6 entries for mo, 6 for tu, etc. To index them makes no sense, or count 0-5 for each day. Don't know how to manage.
Im thinking of store the raw telegram and generate a program json on api command (and maybe as own mqtt topic). We need to modify the api to forward the json to command. For web page this also can not be an entity in the thermostat section, it need a own editor page. I'm asking the question if it is usefill to change the program via ems-esp? I've never changed the program online. Sometimes i use the "pause" function if nobdy is at home for the whole day, but never touched the program.
I agree. It's would be nice to display the timer entries in ems-esp but changing them is not the use-case.
With a JavaScript function in node-red:
The RC310 timers could be display in a node-red dashboard:
The timers can enabled or disabled with the wwmode, wwcircmode, etc. . You can postpone this feature request
And here are the addresses of the vacation timers in the RC310:
vacation1: 000+04:41:20.516 T 386: [emsesp] Me(0x0B) -R-> thermostat(0x10), ?(0x0269), length: 0x19 000+04:41:20.580 N 388: [emsesp] thermostat(0x10) -W-> Me(0x0B), ?(0x0269), data: 18 03 08 18 03 0E 02 11 02 FF 00 00 00 00 00 00 00 FF 00 data 0..2, start: 2024.03.08 ---> 18 03 08 data 3..5, end : 2014.03.14 ---> 18 03 0E data 6, heating mode: 0x01=as Saturday; 0x02=const temp; 0x03= off; 0x04=reduce data 7, heating temp: 0x11 = 17°C data 8, hot water: 0x01=as Saturday; 0x02=off: 0x03=off with disinfect data 9, used for heating: FF=YES | 00=NO data 17,used for hotwater: FF=YES | 00=NO
with a few lines of JS code in node-red it looks like:
what shall we do here @MichaelDvP ?
I don't have these features on my thermostat so not sure how to proceed. I'll remove this issue from the backlog.
I am using a Buderus gas boiler with RC310 thermostat and 2 heating circuits I can decode 4 switchTimes per hc. Each hc has 2 switchPrograms A and B and 2 variants with level and temps. Dhw has another 2 switchPrograms. So in total I have 10 with 2 hc's. (Would be up to 18 with 4 hc's)
For dhw there are 2 switchPrograms: 01FF for warm water buffer load and 0209 for circulation pump - switch between off, eco, hot, on and on, off
The heating circuits can be set to use levels (eco/comfort) or absolute temps per switchTime. These are different telegram types the to be used. (see below all types are raw)
The switch is done by the entity hcx/switchprogmode for each heating circuit between levels and absolute (temps). These entities have been build-in just recently.
switchProgram A: hc1 telegrams level/temps are: 01C3 / 0583 and hc2: 01C4 / 0584 switchProgram B: hc1 telegrams level/temps are: 0349 / 058D and hc2: 034A / 058E
Within #1934 long custom entities in raw are now realized and could be used to maintain the long 84 bytes telegrams. Open questions is where to deconstruct these telegrams and which home automation systems should be supported.
Since quite some time I implemented this in my ioBroker adapter in read/write mode creating the same JSON-objects as the original km200 gateway does. But I needed to implement the right timing for getting the long telegrams and writing them back. It is much better to do this in the EMS-ESP gateway.
Each switchProgram has 6 switchTimes per day, 42 switchTimes per week combined with 42 values. My 10 switchPrograms would create up to 420 mqtt entities if 1:1 implemented and enabled. (2 hc's and dhw with circ. pump)
@tp1de Please use the ems-esp notation for telegram-types. e.g. 0x2FF for dhw1. Thanks.
Can you confirm the value order:
first byte state, second byte time in 15min steps. --- time 0xFF stands for not_set
? (i'm not sure)
first 6 pairs for monday, next 6 for tuesday, etc. (or other day order?) -- are the 6 pairs in row or can only first and last pair be set?
state is for:
dhw1, type 0x2FF, 0x01 for low
, 0x02 for high
--- other states, eg. 0 for off?
circulation dhw1, type 0x309, 0xFF for on
, 0x00 for off
switchtime1 hc1 type 0x2C3, 0x03 for eco
, --- ? for comfort
-- are there other values?
switchtime2 hc1 type 0x683, temperature in 0,5°C steps -- is there min/max?
switchtime 1/2 is selected by program, 0x2B9, offset 11 mode/temperature telegrams (2C3/683) is selected by switchprogmode, offset 19
We can read the whole telegram in ram and generate a json on API command.
For RC35 i have done this a while ago to test. The byte order is different, RC35 have 42 switchtimes and each have also coded the day of week, there is no order in the settings, eg. offset 0/1 can be Fr evening, offset 6/7 mon morning, and 2-5 not set.
Here is the output for ems-esp.local/api/thermostat/hc2/switchtime
RC35_timer.txt
for RC300 i think the json should look
"value":{
"mo":{
"01:{
"time":"07:00","mode":"comfort"
},
"02": {
...
}
},
"tu":{
}
}
For commands we can use the same json structure ans json object in value field. Maybe only a single switchpoint per command, or more, to be discussed if additive or overwriting. to not set values.
But i have no idea
We need a complete concept before starting to code.
Please use the ems-esp notation for telegram-types. e.g. 0x2FF for dhw1. Thanks.
EMS+ and EMS2.0 switchPrograms (For me RC310 and Moduline 2050)
1st step is to select the right telegram type defined by reading the new entities switchprogmodes. If switchprogmode does not exist then use levels for hc.
hc1 - Mode A level 02C3 and absolute temps 0683 (A == program(1)) hc1 - Mode B level 0449 and absolute temps 068D (B == program(2))
hc2 - Mode A level 02C4 and absolute temps 0684 (A == program(1)) hc2 - Mode B level 044A and absolute temps 068E (B == program(2))
dhw1 - Mode A 02FF dhw1 - cp (circulation pump) 0309
Each telegram has a length of 84 bytes: per day there is a maximum of 6 switch points. Each switchpoint is represented by 2 bytes. Therefore each day is 12 bytes: Mo offset 0 0x00 Tu offset 12 0x0C We offset 24 0x18 Th offset 36 0x24 Fr offset 48 0x30 Sa offset 60 0x3C Su offset 72 0x48
Telegram structure of one day looks like this with 2 active switchpoints: P1 P2 P3 P4 P5 P6 ...........................P12 14 01 58 03 FF 03 FF 03 FF 03 FF 03
P1: time of 1st sp: unit 15 minutes --> 0x14 = 20 *15 = 300 minutes = 05:00 hours FF: no active switchpoint
P2 are either levels or absolute temps depending on type. Valid for hc and dhw:
Levels: "FF": "on" / "03": "comfort" / "02": "high" / "01": "eco/low" / "00": "off" temps: Number("0x" + hex) / 2
km200 API JSON uses time in minutes: [ { "dayOfWeek": "Mo", "setpoint": "high", "time": 330 }, { "dayOfWeek": "Mo", "setpoint": "low", "time": 600 }, { "dayOfWeek": "Mo", "setpoint": "high", "time": 1050 }, .......... ]
Within ioBroker I write times: [ { "dayOfWeek": "Mo", "setpoint": "high", "time": "5:30" }, { "dayOfWeek": "Mo", "setpoint": "eco/low", "time": "10:00" }, { "dayOfWeek": "Mo", "setpoint": "high", "time": "17:30" }, .......
I think we do not need the index 01, 02 etc ... in JSON
Important: Any change in switchprogmode will require to re-read different telegram type for hc switchprogram.
The API version with JSON will deliver the similar data object compared to the original Bosch API. Using MQTT for HA I can not identify the right object yet.
At the end JSON structure is not nice to manipulate - neither in ioBroker nor in HA. I am reading the km200 data with node-red as well and create the entities by using mqtt discovery. This works nice and it would be possible to create single entities for all switchpoints. while reading the JSON. Today the switchPrograms are just read-only sensors with JSON-content (text).
Telegram structure of one day looks like this with 2 active switchpoints: P1 P2 P3 P4 P5 P6 ...........................P12 14 01 58 03 FF 03 FF 03 FF 03 FF 03
Ah, ok, first time, second mode. (RC35 have reverse order).
My question is: is this ordered? Or is it possible to have two setpoints this way (second and last set, others unset, last is before second):
FF 03 58 03 FF 03 FF 03 FF 03 14 01
I think we do not need the index 01, 02 etc ... in JSON
If it is not ordered and we want to skip all unset points, we need the number to edit/delete a position. Or we always have to publish/write all unset values.
Levels: "FF": "on" / "03": "comfort" / "02": "high" / "01": "eco/low" / "00": "off"
Does this affect all? Is it really possible to set a cirulation pump to "comfort", or heatingscircuit to "on"? Or (what i think is more logical) does it depend on telegram: hc: comfort/eco, dhw: high/low, circulation on/off?
temps: Number("0x" + hex) / 2
Again: is there a limit? Is it possible to set temp to 3 (1.5°C) or 0xFF (127,5°C)? Or can we use the number to show mode for FF, 3, 2, 1, 0 and temp for all other values.
Ah, ok, first time, second mode. (RC35 have reverse order). My question is: is this ordered? Or is it possible to have two setpoints this way (second and last set, others unset, last is before second):
I have never seen non-ordered telegrams. An index is not necessary and is not used within original Bosch API All unset values use "0xFF" for time and in my case "03" for value. (hc's with level). Temp values can be between 5 (off) and 30.
If it is not ordered and we want to skip all unset points, we need the number to edit/delete a position. Or we always have to publish/write all unset values.
I do always publish a full day if changed (12 bytes including unset points) - So max 7 telegrams But writing just 2 switchPoints (4) bytes should work as well. (if there has been 2 before)
Levels: "FF": "on" / "03": "comfort" / "02": "high" / "01": "eco/low" / "00": "off Does this affect all? Is it really possible to set a cirulation pump to "comfort", or heatingscircuit to "on"? Or (what i think is more logical) does it depend on telegram: hc: comfort/eco, dhw: high/low, circulation on/off?
Decoding for reading telegram is as defined. But possible enum values for writing are: hc level: "eco/low", and "comfort" (for my gas boiler - we might need to check heatpumps maybe "off" as well") dhw (buffer): "off", "eco/low" and "high" dhw cp: "off" and "on"
Again: is there a limit? Is it possible to set temp to 3 (1.5°C) or 0xFF (127,5°C)? Or can we use the number to show mode for FF, 3, 2, 1, 0 and temp for all other values.
Minimum temp is 5°C (off) and maximum is 30°C - the same for manual hc values. Only used for absolute temp telegrams. Absolute temp telegram have a separate type and can be identified easily depending on value of switchprogmode.
The switchtimes do not deliver any values / results yet.
Upps, added the missing register/fetch for the telegrams in 3.7.0-test32
@proddy @MichaelDvP
I implemented a prototype to manage EMS2.0 and EMS+ switchPrograms within Home Assistant using the HACS Scheduler compoment and Node-Red. This solution uses raw hex telegrams send / read by MQTT. It works fine, but was quite tricky for my RC310 thermostat.
I wrote a document about my findings during this process and some analysis I have done on my original Bosch / Buderus gateway. I made a proposal what needs to be done within EMS-ESP to support these entities in a better way. Please have a look. I hope that I have been clear enough ....
I've updated the test build https://github.com/MichaelDvP/EMS-ESP32/releases/tag/test this should give a json structure for
the switchtime1/2 depends on setting of hc1/switchprogmode.
I have not tested writing yet, the implementation is to send a single time in one command as json: "value":{"day":"mo","time":"06:15","mode":"comfort","id":0}
. Writing a complete program needs 42 commands, but editing is much easier.
Also mqtt is not implemented. I think of an extra topic thermostat_swichtimes
as nested or thermostat_switchtime1_hc1
, thermostat_circswitchtime_dhw
, etc as not nested.
I installed your testbuild. 1st test (read) is ok. I would just recomment to keep the modes in English - today the are translated. This makes any logic flow to integrate in UI components (e.g. HASS scheduler) language dependend.
Writing should be per day with up to 6 switchTimes. How do you indicate otherwise deleted switchTimes?
... and may I repeat my recommandation to name the entities switchprograms. The switchtimes are the times to switch within...
I would just recomment to keep the modes in English
Ok
Writing should be per day with up to 6 switchTimes.
Does not work with RC20, RC30 and RC35 where day is coded and can be anywhere in the 42 data, Then i have to write all 42 at once.
How do you indicate otherwise deleted switchTimes?
Reading: values are ignored, writing with {"mode":"not_set",...}
.. and may I repeat my recommandation to name the entities switchprograms. T
We have translated word switchtimes
, so i take this. Renaming and translating is a lot of work. If we have a complete concept and stick to it, we can rename. For now i dont make all the translations.
I've updated, setting should work now.
I've updated, setting should work now.
sorry Michael ... api does not work anymore - no response: message: "no switchtime1 in thermostat"
Oh, sorry, did not mention, renamed the mqtt shortname to switchprog1, etc. as first step.
Is there any reason that you haven't used the active program as well to select the switchprogram? (just switchprogmode) That's the way it is done within Bosch / Buderus App. In this case you would just need switchprog and not 1 and 2.
What is the use of the no (index). It is counted per day?
Is there any reason that you haven't used the active program as well to select the switchprogram?
Yes, we already have two progs for RC35 and i don't want to break this in first step. For RC300 it is better to use onlyactive program, also junkers, that have 6 programs. RC20 is difficult, we know the storage of the program, but not how to select it, it seems only to have predefined programs. RC35 have 11 programs, 2 of them user defined. The selected program is stored at the end of the telegram, so we dont know which program is selected on receiving the first parts of the telegram. Difficult and error prone.
What is the use of the no (index). It is counted per day?
Yes, for RC300 and Junkers it counts 0..5, for RC20, RC30 andRC35 it counts 0..42. I rename to "no" to avoid confusion with id (hc).
Thanks for your explanations - all the thermostat variants make it quite complex.
Let me share my observations: I recognized that all telegram types for switchPrograms are having read-requests every minute (even the inactive ones). This increases the TX reads per 24hours from approx. 42.000 to 98.000. Can't this be done in a different way? (similar to original gateways - I tried to explain in the PDF)
In respect to the JSON structure and the no (index), I ask myself how to track changes in home automation systems and how to prepare the post commands. Without tracking changes of all 42 switchTimes have to be send by single http posts. What kind of timedelay is needed inbetween?
Wouldn't it be easier by indicating in no the actual telegram position for all devices (for RC300/310 and Junkers too) and to send the full JSON in one http post?
I've updated the test build: https://github.com/MichaelDvP/EMS-ESP32/releases
{"day":"mo","no":2,"time":"12:00","mode":"off"}
for a single time, [{"day":"mo","no":2,"time":"12:00","mode":"on"},{"day":"mo","no":3,"time":"12:30","mode":"off"}]
for two settings at once. With array command always the complete 84byte telegram is written.thermostat_switchprog
, mqtt commands as usual to topic thermostat.I've tested only with my RC35. Because RC300 and Junkers are different there may be some issues.
I tested on my RC310: (api gets first)
dhw entities are working for api reads. I just recognized that you kept the numbering per day - any reason why? But the hc switchprog returns an empty array - no values. Please check.
Edit:
the hc switchprograms just contain values when I send read requests for full telegram or post the values from HA. (raw telegrams) Then "level" contains data. "absolute" does not work.
I just recognized that you kept the numbering per day - any reason why?
I need them for RC20/30/35 and for deleting/changing only a single value.
But the hc switchprog returns an empty array - no values. Please check.
Please make a full log to see if i missed to fetch a telegram. But after start it needs some minutes, wait for hc active, first fetch to get the active program, second fetch to get the values.
Edit: found and fixed a typo for 1st absolute program, second absolute should work. Please update (in a few minutes, after github action has finished compiling)
Edit: found and fixed a typo for 1st absolute program, second absolute should work. Please update (in a few minutes, after github action has finished compiling)
After installing your last version reading seems to work for "level" and "absolute" and values are initialized in the right way after approx 90 to 100 seconds after start. Changing switchprogmode or program is working as well.
I just recognized that you kept the numbering per day - any reason why? I need them for RC20/30/35 and for deleting/changing only a single value.
I thought that for RC20-35 you will use the absolute position within telegram? (if yes, this could be done for EMS+ as well)
(if yes, this could be done for EMS+ as well)
I think it makes no sense. For RC35 a value "no":35, "day":"mo", "time":"00:30"
is valid, for RC300 each parameter is in range, but the combination is not valid because the day is asociated with number. So it's more clear to count the values within the day 0..5.
BTW: you can also use numbers for the json parameters. e.g. "day":0 instead of "day":"mo" or "time":135 instead of "time":"02:15"
I checked within my running ioBroker instance and HA.
Within MQTT you have created one new element ems-esp/thermostat_switchprog containing all switchprograms. This is nice, since I could create the HA entities - e.g. scheduler entities - from this data. But how do changes would work - updating the whole structure?
The ioBroker adapter reads the device value list calling .../api/thermostat. The switchprog entities are there but with an empty JSON: {} Would be good to get the data content in as well to test if the adapter can be used without changes. (I hope so)
Within MQTT you have created one new element ems-esp/thermostat_switchprog containing all switchprograms. This is nice, since I could create the HA entities - e.g. scheduler entities - from this data. But how do changes would work - updating the whole structure?
The topic is published in intervals of "other", fetch on the bus is every minute. Writing multiple points at once updates the buffer direct and sends the buffer to the bus. For single points the buffer update is actually on the next fetch. Mqtt publish on change is not yet implemented for the programs.
The ioBroker adapter reads the device value list calling .../api/thermostat. The switchprog entities are there but with an empty JSON: {}
Yes, i added this to tell your adapter that there is a value (and it's json). I think the response for RC300 with some hc is getting too big and blow the ram of chips without PSRAM.
Would be good to get the data content in as well to test if the adapter can be used without changes. (I hope so)
I think the json data needs a special handling, so it also can be fetched seperately.
You can GET /api/thermostat/hc1/switchprog/value
If you have a S32 or older gateway (ESP32 no PSRAM), try to get api/thermostat/entities
Is it complete? (but afair you have only one hc).
I am using an E32 V2.
api/thermostat/entities shows all data including switchprograms.
If you are concerned about memory allocation issues, than I need to introduce a separate call for each switchprogram to get the data. This would need adjustments to the ioBroker adapter. On the other side PSRAM (and free PSRAM) is known. Couldn't you add json content depending on PSRAM value?
Any chance to get a test version with data?
If I need to do changes on the adapter anyhow, then it might be better to keep the empty JSON.
Any chance to get a test version with data?
Sure, it's a test version, we can add and remove something to test. I've added to api/thermostat. For now for all, A Psram check is included, but commented out. Wait 10 min for compile done in github.
Thanks Michael. To work with ioBroker object browser the json content needs to be stringified. If your plan to enable it would be nice to change the coding. Otherwise I need to adjust the iobroker adapter.
To work with ioBroker object browser the json content needs to be stringified.
Ok, done.
Nice job. Everything is working fine. No adjustments are needed for the iobroker adapter. Changing values are working as well.
cool! I'll take a look as well. Shall we squeeze this into 3.7.0 or release it after in a dev 3.7.1 when people have tested it? It's been 6+ months since the last stable release and BBQKees is still using 3.6.5 on his latest batch
I will continue testing today. I do have 2 questions:
I have updated to latest dev34-changes and refactored some parts of the switchprog code, hopefully without braking something. Please check.
shall we avoid the backslashes within json stringify on .../api/thermostat call ? (just for nicer read)
The backslashes are needed for json-strings with inner quotationmarks. To avoid i have to send as json-array again. In api/thermostat/switchprog
it's a json, in api/thermostat/switchprog/value
a string not inside a json.
The switchprograms are not within HA entities
I don't use HA and have no idea how to add this to HA. Also i think this will generate hundreds of mqtt-config messages. But we can try if you tell me what is needed for a HA config.
@proddy
Shall we squeeze this into 3.7.0 or release it after in a dev 3.7.1
There is also Junkers switchprog that needs testing. I think i've also broke some modbus assignes, can't get the scripts running on first try. For the switchprog there is also no idea how to integrate in web-UI, now it's just like a command to change values, but no display for the program. BTW: RC20 switchprog should also work with telegram 0x8F, RC20_N not, don't know what telegram is used., maybe same as RC30/35? (0x3F).
If you want to release soon, than better merge this to 3.7.1.
I have updated to latest dev34-changes and refactored some parts of the switchprog code, hopefully without braking something. Please check.
Works still fine.
Json stringify is fine for me and reflects the json standards.
For HA I think creating entities from EMS-ESP will work only with discovery and putting the switchprogram json into the attributes. Since the HASS scheduler is using an own entity storage these schedules have to be created by using the scheduler service calls. This can be done by reading the mqtt data. I do not see any need to go for ha discovery in parallel, unless @proddy has different ideas.
@MichaelDvP I tested on my second installation: my Dutch holidayhouse with Moduline 2050 EMS 2.0 thermostat.
This thermostat does not have any switchprogmode and just one program. It is permanent "level" and prog1 for hc1.
In this case no switchprogram is discovered. I think this might be caused by the non-existing switchprogmode.
There is no value but by calling per api it is shown, but without index. (it should not exist)
The Moduline 2050 has deviceID: 0x18 and productId: 157 / version 43.03 switchprogmode is not shown due to missing index. This is ok, but then you should use "level". For program you show both prog1 and prog2, but prog 2 does not exist.
I've removed program and switchprogmode from RC100 compatible thermostats and switchprog-default to first telegram.
I've removed program and switchprogmode from RC100 compatible thermostats and switchprog-default to first telegram.
Yes it works now.
On my RC310 system I was testing my scheduler scripts now adjusted to the new switchprograms. I am getting error messages (not always) when I change the schedule/switchprograms. I am using 2 schedules: one for workdays (mo-fr) and one for weekend (sa-su). The changes are send per schedule when saved using days with numbers. E.g.
2024-09-07 14:57:58.769 I 69: [command] Called command thermostat/switchprog (Programm Schaltzeit) with value [{"day":0,"time":"06:15","mode":"comfort","no":0},{"day":0,"time":"21:30","mode":"eco","no":1},{"day":1,"time":"06:15","mode":"comfort","no":0},{"day":1,"time":"21:30","mode":"eco","no" :1 ..... 2024-09-07 14:57:59.295 E 70: [emsesp] Last Tx write rejected by host 2024-09-07 14:58:01.743 E 71: [emsesp] Last Tx write rejected by host 2024-09-07 14:58:02.545 E 72: [emsesp] Last Tx write rejected by host
At the end the changes are done but the mqtt entity ems-esp/thermostat/switch_prog disapears for approx. 1-2 minutes. I have got one crash of EMS-ESP without reboot. WiFi was lost and accesspoint down as well. I needed a hard reset. Looks like a memory problem.
What are the constraints of sending changes?
Last Tx write rejected by host
This is from the bus, that does not accept the write. I send the 84 bytes in 4 parts. I've reordert to start with offset 0, maybe this helps. Please make a full log and trigger this errormessage to see what part is rejected and if the retry is accepted.
Looks like a memory problem.
I forgot to check valid no-values for RC300, was the no
greater 5 for sunday?
Discussed in https://github.com/emsesp/EMS-ESP32/discussions/1593