Upgrade to 2.1.0b7 and missing mixer circuit

[PhillyGilly]

Upgraded to 1.1.0b7 last night and ran into a few problems. Now that I have auto-discovery almost nothing is working in HA as I had but all the "Friendly Names" in my sensor.yaml file. AFAIK the way to do this is to edit configuration.yaml homeassistant:

other stuff may be here

customize: !include customize.yaml

and in customize.yaml: sensor.warm_water_activated friendly_name: 'DHW activated'

I now have 200+ entities so I am looking forward to a good few hours getting back on track, although I know this will be a good investment. But is there a better way?

More importantly I have lost one of my mixer circuits that was present and working under 2.0.2b0. The ESP is seeing the device but not generating the MQTT topic.

[bbqkees]

Not sure why but when I updated last week from one beta to the next some mqtt topics were not published anymore. After a hard power off they appeared again.

[PhillyGilly]

Not sure why but when I updated last week from one beta to the next some mqtt topics were not published anymore. After a hard power off they appeared again.

@bbqkees Thanks. But that didn't work. The data in the EMS webpage is the same as in the Sense II controller.

[MichaelDvP]

The mixing mqtt is changed to nested format but not tested in HA discovery (i do not have HA, proddy don't have a mixer). Look at he discussion in #549 and check what mqtt output you get for mixer and discovery.

[PhillyGilly]

Hi @MichaelDvP. Thanks.

I'm not getting nested data in MQTT explorer just one Topic: mixing_data = {"type":"wwc","wwTemp":42.2,"pumpStatus":"off","tempStatus":1}

Under v2.0 I was getting two Topics:: mixing_data9 = {"type":"wwc", "wwTemp":53.2, "pumpStatus":0,"tempStatus" 0} mixing_data10 = {"type":"wwc","wwTemp":58.4,"pumpStatus":0,"tempStatus":0}

I seem to recall at the very beginning of the development of the MM200 module there was some discussion about using nested payloads, but I think this was discounted because the data was coming in on separate EMS telegrams.

I have also had a little dig under the HA bonnet and after upgrading to the latest release I found there are new tools to deal with Entity names (very pleased) and tools to deal with Discovery.

Although the MM10 module is there in #549, I have nothing in HA discovery for my MM200. However when I look at the Home Assistant instance in MQTT Explorer, I can see:

mixing_wwc1_wwTemp config = {"name":"Current warm water temperature","uniq_id":"mixing_wwc1_wwTemp","unit_of_meas":"°C","stat_t":"ems-esp/mixing_data","val_tpl":"{{value_json.wwc1.wwTemp}}","ic":"mdi:coolant-temperature","dev":{"ids":["ems-esp-mixing"]}} mixing_wwc1_pumpStatus config = {"name":"Current pump status","uniq_id":"mixing_wwc1_pumpStatus","stat_t":"ems-esp/mixing_data","val_tpl":"{{value_json.wwc1.pumpStatus}}","dev":{"ids":["ems-esp-mixing"]}} mixing_wwc1_tempStatus config = {"name":"Current temperature status","uniq_id":"mixing_wwc1_tempStatus","stat_t":"ems-esp/mixing_data","val_tpl":"{{value_json.wwc1.tempStatus}}","dev":{"ids":["ems-esp-mixing"]}}

mixing_wwc2_wwTemp config = {"name":"Current warm water temperature","uniq_id":"mixing_wwc2_wwTemp","unit_of_meas":"°C","stat_t":"ems-esp/mixing_data","val_tpl":"{{value_json.wwc2.wwTemp}}","ic":"mdi:coolant-temperature","dev":{"ids":["ems-esp-mixing"]}} mixing_wwc2_pumpStatus config = {"name":"Current pump status","uniq_id":"mixing_wwc2_pumpStatus","stat_t":"ems-esp/mixing_data","val_tpl":"{{value_json.wwc2.pumpStatus}}","dev":{"ids":["ems-esp-mixing"]}} mixing_wwc2_tempStatus config = {"name":"Current temperature status","uniq_id":"mixing_wwc2_tempStatus","stat_t":"ems-esp/mixing_data","val_tpl":"{{value_json.wwc2.tempStatus}}","dev":{"ids":["ems-esp-mixing"]}}

Not sure what that means.

[MichaelDvP]

mixing_data = {"type":"wwc","wwTemp":42.2,"pumpStatus":"off","tempStatus":1}

Here is a bug, try in a few minutes a new build.

[PhillyGilly]

mixing_data = {"type":"wwc","wwTemp":42.2,"pumpStatus":"off","tempStatus":1}

Here is a bug, try in a few minutes a new build.

I found the code change thanks, but not the 8266 bin file.

[MichaelDvP]

Here

[glitter-ball]

I need to take the auto-discovery plunge at some point but need to think it through first. Is there any wiki or other info on how best to do it?

Wondering if I should hang on a bit longer to let the new release bed in a bit?

[suysh]

I'm already started to implement the new structure in my code. (maybe a bad moment for me to upgrade, but this may help to solve). I downloaded the version from the link 2 posts above. Now the topics ems-esp/mixing_data1 and ems-esp/mixing_data2 and ems-esp/boiler_data are not publishing any more. I'm getting publishing errors in the mqtt dashboard.

[MichaelDvP]

In v2.1 the mixing topics depends on mqtt-format: nested and HA publishes a message for all mixers ems-esp/mixing_data = {"hc1":{...}} single publishes a message per circuit like ems-esp/mixing_data1 = {"type":"hc1", ...} , ems-esp/mixing_data2 = ...

[suysh]

Aha, indeed I receive all mixing data on the topic ems-esp/mixing_data. (confusing documentation, but is normal due to upgrade) Can you check why I'm not receiving data on the topic ems-esp/boiler_data ?

[suysh]

Update, I'm missing the pumpMod value in the structure as be present before. => valveStatus is now the pumpMod from before and indicates the percentage of mixing. = mismatch between documentation

[PhillyGilly]

I need to take the auto-discovery plunge at some point but need to think it through first. Is there any wiki or other info on how best to do it?

Wondering if I should hang on a bit longer to let the new release bed in a bit?

I'm not really HA expert, but I seem to have found a half reasonable way to do this migration. It all looked a bit daunting but the tools in the latest version of HA make it fairly easy. The biggest challenge is with "Friendly names". I only displayed the entities that are accessible via the original boiler display and named them using the the British Worcester/Bosch manual. I also had value-topic: if-then to convert codes to text and string manipulation convert seconds to hours both of which need figuring out. I'll share thoughts on the rest of the process as soon as I've got the MM200 working and finished the migration.

[MichaelDvP]

@PhillyGilly: Does the mqtt-format HA work now with HA discovery?

[PhillyGilly]

@PhillyGilly: Does the mqtt-format HA work now with HA discovery?

Sorry @MichaelDvP, are you asking if latest version of HA is discovering the ESP with its mqtt format set to HA? Yes that's working although I haven't uploaded your latest firmware yet. I will do that and feedback later today.

[PhillyGilly]

@MichaelDvP Very good, thank you. MQTT explorer reports: mixing_data = {"wwc1":{"wwTemp":50.1,"pumpStatus":"off","tempStatus":1},"wwc2":{"wwTemp":35.6,"pumpStatus":"off","tempStatus":1}} The device detection on the Dashboard didn't work on restart and I had to power cycle the ESP. I haven't checked if this is working in HA yet.

[PhillyGilly]

Initial feedback from HA is:

1. New device (current warm water) is auto discovered
2. New device contains entities that aren't in the mqtt topic However the entities that I want/need are there and usable.

[proddy]

I think I know where the problem is. I'll see if I can fix it.

[PhillyGilly]

Update, I'm missing the pumpMod value in the structure as be present before. => valveStatus is now the pumpMod from before and indicates the percentage of mixing. = mismatch between documentation

I've managed by a bit of trial and error to match up most of my auto-discovered entities to previous sensor.yaml entities. This are the ones that are missing or causing me problems, the rest I can live with:

32 sensor.boiler_pump_modulation_max

name: 'Boiler Pump Modulation Max' unit_of_measurement: '%' value_template: '{{ value_json.pump_mod_max }}'

33 sensor.boiler_pump_modulation_min

name: 'Boiler Pump Modulation Min' unit_of_measurement: '%' value_template: '{{ value_json.pump_mod_min }}'

43 sensor.gas_burner_working_minutes

name: 'Gas Burner Working Time' unit_of_measurement: 'hr' value_template: > {% set mins = value_json.burnWorkMin %} {% set hours = ((mins | int / 60) | string).split('.')[0] %} {{hours}}

44 sensor.ch_working_minutes

name: 'CH Working Time' unit_of_measurement: 'hr' value_template: > {% set mins = value_json.heatWorkMin %} {% set hours = ((mins | int / 60) | string).split('.')[0] %} {{hours}}

46 sensor.service_code_number

name: 'Service Code Number' unit_of_measurement: '' icon: 'mdi:power-cycle' value_template: "{% if value_json.serviceCodeNumber == 200 %} CH active {% elif value_json.serviceCodeNumber == 201 %} HW active {% elif value_json.serviceCodeNumber == 202 %} CH anti cycle {% elif value_json.serviceCodeNumber == 203 %} Stand by {% elif value_json.serviceCodeNumber == 204 %} CH cooling {% elif value_json.serviceCodeNumber == 208 %} Service test {% elif value_json.serviceCodeNumber == 265 %} Low CH load {% elif value_json.serviceCodeNumber == 268 %} Component test {% elif value_json.serviceCodeNumber == 270 %} Power up {% elif value_json.serviceCodeNumber == 283 %} Burner start {% elif value_json.serviceCodeNumber == 284 %} Ignition {% elif value_json.serviceCodeNumber == 305 %} HW anti cycle {% elif value_json.serviceCodeNumber == 357 %} Air purge {% elif value_json.serviceCodeNumber == 358 %} Valve kick {% else %} {{ value_json.serviceCodeNumber }} {% endif %}"

98 sensor.uba_up_time

name: 'UBA Uptime' unit_of_measurement: 'hr' value_template: > {% set secs = value_json.UBAuptime %} {% set hours = ((secs | int / 3600) | string).split('.')[0] %} {{hours}}

FWIW here is the payload from Jan 2020 that I used when creating the sensor.yaml Topic:boiler_data Payload:{"wWComfort":"Hot","wWSelTemp":60,"wWDesinfectionTemp":70,"selFlowTemp":48,"selBurnPow":100,"curBurnPow":36,"pumpMod":96,"wWCircPump":0,"outdoorTemp":11.6,"wWCurTmp":29.3,"wWCurFlow":0,"curFlowTemp":47.8,"switchTemp":0,"wWActivated":"on","wWOnetime":"off","burnGas":"on","flameCurr":20.1,"heatPmp":"on","fanWork":"on","ignWork":"off","wWCirc":"off","heating_temp":52,"pump_mod_max":100,"pump_mod_min":60,"wWHeat":"off","wWStarts":998,"wWWorkM":10620,"UBAuptime":985772,"burnStarts":24054,"burnWorkMin":149491,"heatWorkMin":138871,"ServiceCode":"","ServiceCodeNumber":200}

[PhillyGilly]

I need to take the auto-discovery plunge at some point but need to think it through first. Is there any wiki or other info on how best to do it? Wondering if I should hang on a bit longer to let the new release bed in a bit?

I'm not really HA expert, but I seem to have found a half reasonable way to do this migration. It all looked a bit daunting but the tools in the latest version of HA make it fairly easy. The biggest challenge is with "Friendly names". I only displayed the entities that are accessible via the original boiler display and named them using the the British Worcester/Bosch manual. I also had value-topic: if-then to convert codes to text and string manipulation convert seconds to hours both of which need figuring out. I'll share thoughts on the rest of the process as soon as I've got the MM200 working and finished the migration.

@glitter-ball this should help https://emsesp.github.io/docs/#/Home-Assistant plus my recent comment to Proddy in the community.

[proddy]

@PhillyGilly that's a lot of work you put it into making it your own in HA. You took the plunge, nice! If you have better icons and wording we can change that too. I think even the service codes we could map to text at some point. The less .yaml the better.

[glitter-ball]

Interesting! Am I right in thinking all the basic mapping is done by auto discovery but I can still change formatting, icons, templates etc. in HA? Or is the aim to avoid as much YAML as possible and do most of the heavy lifting in EMS-ESP?

I suppose I should just take a snapshot and have a play...

[PhillyGilly]

Interesting! Am I right in thinking all the basic mapping is done by auto discovery but I can still change formatting, icons, templates etc. in HA? Or is the aim to avoid as much YAML as possible and do most of the heavy lifting in EMS-ESP?

I suppose I should just take a snapshot and have a play...

Yes. You're right. Follow the Wiki and see my comments to Proddy last night in the forum on how to tweak. There are some entities where I manipulated the json value in my sensor.yamk file eg converting time from seconds to hours or alarm codes to text that I haven't worked out yet. But my sensor file has reduced from nearly 1,000 lines to less than 100.

[proddy]

@glitter-ball you can override everything in HA, either directly via the HA web interface or using .yaml files. The naming and icons are just defaults in EMS-ESP but I didn't spend a lot of time thinking about them. If you have better naming or better icons ideas let me know and I'll make the code changes,

[glitter-ball]

@proddy and @PhillyGilly - thanks for your tips. Doesn't sound too painful, so I'll maybe give it a go next weekend!

Naming and icons is always going to be a personal thing - my current arrangement below. Happy to share the .yaml if anything catches your eye but I've seen some pretty slick stuff on here from others too *8)

Part 1

Part 2

[glitter-ball]

I also need to fix the thermostat climate card which screams every minute when I put the heating in manual mode...

2020-10-16 09:22:12 ERROR (MainThread) [homeassistant.components.mqtt.climate] Invalid modes mode: manual

[proddy]

@glitter-ball I may steal some of those mdi icons. Can you share which ones you used?

[glitter-ball]

OK. My rather untidy sensor.yaml is here. Hopefully you can pull out icon refs from it!

- platform: mqtt
  state_topic: 'ems-esp/thermostat_data1'
  name: 'Current Mode'
  value_template: "{{ value_json.mode }}" 

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Gas'
  icon: 'mdi:fire'
  value_template: '{{ value_json.burnGas }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Fan'
  icon: 'mdi:fan'
  value_template: '{{ value_json.fanWork }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Ignition'
  icon: 'mdi:flash'
  value_template: '{{ value_json.ignWork }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Pump'
  icon: 'mdi:water-pump'
  value_template: '{{ value_json.heatPump }}'  

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Outside Temperature'
  unit_of_measurement: '°C'
  icon: 'mdi:home-export-outline'
  value_template: '{{ value_json.outdoorTemp }}'

- platform: mqtt
  state_topic: 'ems-esp/thermostat_data1'
  name: 'Room Temperature'
  unit_of_measurement: '°C'
  icon: 'mdi:home-import-outline'
  value_template: "{{ value_json.currtemp }}"

- platform: mqtt
  state_topic: 'ems-esp/thermostat_data1'
  name: 'Set Temperature'
  unit_of_measurement: '°C'
  icon: 'mdi:car-cruise-control'
  value_template: "{{ value_json.seltemp }}" 

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Flow temp'
  unit_of_measurement: '°C'
  value_template: '{{ value_json.curFlowTemp }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'HW set temperature'
  unit_of_measurement: '°C'
  icon: 'mdi:car-cruise-control'
  value_template: '{{ value_json.wWSelTemp }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'HW current temperature'
  unit_of_measurement: '°C'
  value_template: '{{ value_json.wWCurTmp }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Heating'
  value_template: '{{ value_json.heatingActivated }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Hot Water activated'
  value_template: '{{ value_json.wWActivated }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: '3-way valve'
  icon: 'mdi:valve'
  value_template: "{% if value_json.wWHeat == 'off' %} CH {% else %} HW {% endif %}"

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Burner power'
  unit_of_measurement: '%'
  icon: 'mdi:fire'
  value_template: '{{ value_json.curBurnPow }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Pump speed'
  icon: 'mdi:water-pump'
  unit_of_measurement: '%'
  value_template: '{{ value_json.pumpMod }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'wWComfort'
  value_template: '{{ value_json.wWComfort }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Boiler setpoint'
  unit_of_measurement: '°C'
  icon: 'mdi:car-cruise-control'
  value_template: '{{ value_json.selFlowTemp }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Burner setpoint'
  unit_of_measurement: '%'
  value_template: '{{ value_json.selBurnPow }}'  

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Ionisation current'
  unit_of_measurement: 'µA'
  icon: 'mdi:flash'
  value_template: '{{ value_json.flameCurr }}'

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Service Code'
  unit_of_measurement: ''
  value_template: '{{ value_json.serviceCodeNumber }}' 

- platform: mqtt
  state_topic: 'ems-esp/boiler_data'
  name: 'Status'
  unit_of_measurement: ''
  icon: 'mdi:power-cycle'
#  value_template: '{{ value_json.serviceCodeNumber }}'
  value_template: "{% if value_json.serviceCodeNumber == 200 %} CH active 
    {% elif value_json.serviceCodeNumber == 201 %} HW active
    {% elif value_json.serviceCodeNumber == 202 %} CH anti cycle
    {% elif value_json.serviceCodeNumber == 203 %} Stand by
    {% elif value_json.serviceCodeNumber == 204 %} CH cooling
    {% elif value_json.serviceCodeNumber == 208 %} Service test
    {% elif value_json.serviceCodeNumber == 265 %} Low CH load
    {% elif value_json.serviceCodeNumber == 268 %} Component test
    {% elif value_json.serviceCodeNumber == 270 %} Power up
    {% elif value_json.serviceCodeNumber == 283 %} Burner start
    {% elif value_json.serviceCodeNumber == 284 %} Ignition
    {% elif value_json.serviceCodeNumber == 305 %} HW anti cycle
    {% elif value_json.serviceCodeNumber == 357 %} Air purge
    {% elif value_json.serviceCodeNumber == 358 %} Valve kick
    {% else %} {{ value_json.serviceCodeNumber }} {% endif %}"

# general

- platform: mqtt
  state_topic: 'ems-esp/status'
  name: 'ems-esp status'

###################################################
# EMS-ESP Statistics - heartbeat
###################################################
- platform: mqtt
  state_topic: 'ems-esp/heartbeat'
  name: 'ems_esp_wifi'
  unit_of_measurement: '%'
  value_template: '{{ value_json.rssid }}'
  qos: 1
  availability_topic: 'ems-esp/status'
  payload_available: "online"
  payload_not_available: "offline"

- platform: mqtt
  state_topic: 'ems-esp/heartbeat'
  name: 'ems_esp_load'
  unit_of_measurement: '%'
  value_template: '{{ value_json.load }}'
  qos: 1
  availability_topic: 'ems-esp/status'
  payload_available: "online"
  payload_not_available: "offline"

- platform: mqtt
  state_topic: 'ems-esp/heartbeat'
  name: 'ems_esp_freemem'
  unit_of_measurement: '%'
  value_template: '{{ value_json.freemem }}'
  qos: 1
  availability_topic: 'ems-esp/status'
  payload_available: "online"
  payload_not_available: "offline"

- platform: mqtt
  state_topic: 'ems-esp/heartbeat'
  name: 'ems_esp_uptime'
  unit_of_measurement: 's'
  value_template: '{{ value_json.uptime }}'
  qos: 1
  availability_topic: 'ems-esp/status'
  payload_available: "online"
  payload_not_available: "offline"

- platform: mqtt
  state_topic: 'ems-esp/heartbeat'
  name: 'MQTT publish fails'
  value_template: '{{ value_json.mqttpublishfails }}'
  qos: 1
  availability_topic: 'ems-esp/status'
  payload_available: "online"
  payload_not_available: "offline"

- platform: mqtt
  state_topic: 'ems-esp/sensor_28-BE2A-7791-0402-65'
  name: 'Return temp'
  unit_of_measurement: '°C'
  value_template: '{{ value_json.temp | round(1) }}'

- platform: template
  sensors:
    differential:
      friendly_name: "Flow-Ret diff"
      unit_of_measurement: '°C'
      icon_template: 'mdi:format-align-middle'
      value_template: "{{ (states('sensor.flow_temp') | float - states('sensor.return_temp') | float) | round(1) }}"

- platform: systemmonitor
  resources:
      - type: disk_use_percent
      - type: memory_use_percent
      - type: processor_use
      - type: processor_temperature

[proddy]

Update, I'm missing the pumpMod value in the structure as be present before. => valveStatus is now the pumpMod from before and indicates the percentage of mixing. = mismatch between documentation

is this still the case?

I've managed by a bit of trial and error to match up most of my auto-discovered entities to previous sensor.yaml entities. This are the ones that are missing or causing me problems, the rest I can live with:

32 sensor.boiler_pump_modulation_max

name: 'Boiler Pump Modulation Max' unit_of_measurement: '%' value_template: '{{ value_json.pump_mod_max }}'

33 sensor.boiler_pump_modulation_min

name: 'Boiler Pump Modulation Min' unit_of_measurement: '%' value_template: '{{ value_json.pump_mod_min }}'

43 sensor.gas_burner_working_minutes

name: 'Gas Burner Working Time' unit_of_measurement: 'hr' value_template: > {% set mins = value_json.burnWorkMin %} {% set hours = ((mins | int / 60) | string).split('.')[0] %} {{hours}}

44 sensor.ch_working_minutes

name: 'CH Working Time' unit_of_measurement: 'hr' value_template: > {% set mins = value_json.heatWorkMin %} {% set hours = ((mins | int / 60) | string).split('.')[0] %} {{hours}}

46 sensor.service_code_number

name: 'Service Code Number' unit_of_measurement: '' icon: 'mdi:power-cycle' value_template: "{% if value_json.serviceCodeNumber == 200 %} CH active {% elif value_json.serviceCodeNumber == 201 %} HW active {% elif value_json.serviceCodeNumber == 202 %} CH anti cycle {% elif value_json.serviceCodeNumber == 203 %} Stand by {% elif value_json.serviceCodeNumber == 204 %} CH cooling {% elif value_json.serviceCodeNumber == 208 %} Service test {% elif value_json.serviceCodeNumber == 265 %} Low CH load {% elif value_json.serviceCodeNumber == 268 %} Component test {% elif value_json.serviceCodeNumber == 270 %} Power up {% elif value_json.serviceCodeNumber == 283 %} Burner start {% elif value_json.serviceCodeNumber == 284 %} Ignition {% elif value_json.serviceCodeNumber == 305 %} HW anti cycle {% elif value_json.serviceCodeNumber == 357 %} Air purge {% elif value_json.serviceCodeNumber == 358 %} Valve kick {% else %} {{ value_json.serviceCodeNumber }} {% endif %}"

98 sensor.uba_up_time

name: 'UBA Uptime' unit_of_measurement: 'hr' value_template: > {% set secs = value_json.UBAuptime %} {% set hours = ((secs | int / 3600) | string).split('.')[0] %} {{hours}}

is there anything that you'd like to see EMS-ESP do to format these values?

[proddy]

I made some changes to the Mixer unit for HA in 2.0.1b11. Check it out and let know if it works. I don't have a mixing unit so its hard to test,

[proddy]

closing this issue. the mixer circuits are identified. If new issues pop up we can open a new issue.