Advise on updating the dsmr component

[afarago]

The problem

DSMR used by Hungarian energy provider in Budapest capital (ELMŰ/EON/MVM) is not covered and partly incompatible with the esphome dsmr component.

Also the component is based on a library that seems to be off and is tough to correct.

Which version of ESPHome has the issue?

2022.2.3

What type of installation are you using?

pip

Which version of Home Assistant has the issue?

2022.2.9

What platform are you using?

ESP8266

Board

esp12 in https://www.zuidwijk.com/product/slimmelezer-plus/

Component causing the issue

dsmr

Example YAML snippet

dsmr:
  id: dsmr_instance
  crc_check: false
  max_telegram_length: 3000

Anything in the logs that might be useful for us?

sensor:
  - platform: dsmr
    energy_delivered:
      name: "${device_friendly_name} Energy Consumed"
    energy_delivered_tariff1:
      name: "${device_friendly_name} Energy Consumed Tariff 1"
    energy_delivered_tariff2:
      name: "${device_friendly_name} Energy Consumed Tariff 2"
    energy_returned:
      name: "${device_friendly_name} Energy Produced"
    energy_returned_tariff1:
      name: "${device_friendly_name} Energy Produced Tariff 1"
    energy_returned_tariff2:
      name: "${device_friendly_name} Energy Produced Tariff 2"
    power_delivered:
      name: "${device_friendly_name} Power Consumed"
    power_returned:
      name: "${device_friendly_name} Power Produced"
    # electricity_failures:
    #   name: "Electricity Failures"
    #   icon: mdi:alert
    # electricity_long_failures:
    #   name: "Long Electricity Failures"
    #   icon: mdi:alert
    voltage_l1:
      name: "${device_friendly_name} Voltage Phase 1"
    voltage_l2:
      name: "${device_friendly_name} Voltage Phase 2"
    voltage_l3:
      name: "${device_friendly_name} Voltage Phase 3"
    current_l1:
      name: "${device_friendly_name} Current Phase 1"
    current_l2:
      name: "${device_friendly_name} Current Phase 2"
    current_l3:
      name: "${device_friendly_name} Current Phase 3"
    electricity_threshold:
      name: "${device_friendly_name} Electricity Threshold"
    electricity_tariff:
      name: "${device_friendly_name} Electricity Tariff"
    # reactive_power_qi:
    #   name: "${device_friendly_name} Reactive Power QI"
    # reactive_power_qii:
    #   name: "${device_friendly_name} Reactive Power QII"
    # reactive_power_qiii:
    #   name: "${device_friendly_name} Reactive Power QIII"
    # reactive_power_qiv:
    #   name: "${device_friendly_name} Reactive Power QIV"
    # reactive_energy_qi:
    #   name: "${device_friendly_name} Reactive Energy QI"
    # reactive_energy_qii:
    #   name: "${device_friendly_name} Reactive Energy QII"
    # reactive_energy_qiii:
    #   name: "${device_friendly_name} Reactive Energy QIII"
    # reactive_energy_qiv:
    #   name: "${device_friendly_name} Reactive Energy QIV"
    # instantaneous_power_factor:
    #   name: "${device_friendly_name} Instantaneous Power Factor"
    # instantaneous_power_factor_l1:
    #   name: "${device_friendly_name} Instantaneous Power Factor Phase 1"
    # instantaneous_power_factor_l2:
    #   name: "${device_friendly_name} Instantaneous Power Factor Phase 2"
    # instantaneous_power_factor_l3:
    #   name: "${device_friendly_name} Instantaneous Power Factor Phase 3"
    frequency:
      name: "${device_friendly_name} Frequency"
    # absolute_active_energy:
    #   name: "${device_friendly_name} Absolute active energy"

text_sensor:
  - platform: dsmr
    identification:
      name: "${device_friendly_name} Identification"
    breaker_status:
      name: "${device_friendly_name} Breaker Status"

Additional information

HELP NEEDED: 1) I have created a fork also duplicating the dsmr library to add the codes according to the standard / Hungarian. The code itself is ugly and I have absolutely no idea on how to format that better or structure that would be at least acceptable, but more likely nice to submit a PR. Please advise me on possible architecture so that I can contribute accordingly for the sake of the community.

Attached pdf, page 20. EON-leiras-az-ugyfelek-szamara-SX6X1-S12U16-S34U18-V010.pdf

My forked github repo: https://github.com/afarago/dsmr

2) Additional help needed: "energy_delivered_lux" sensor name is misleading and is not according to the DSMR standard - thus should be removed or changed to the appropriate "Positive active energy (A+) total" or at least "energy_delivered". How to remove this without breaking backwards compatibility.

BACKGROUND: I believe that the dsmr component is both having consistency issues and also is lack of many DSMR standard measurement codes. Nevertheless it is an awesome piece of work and happy to have it.

I see that the dsmr component uses the arduino dsmr library as a dependency. The library codes state some _lux codes in a misleading format, and though I understand that it emerged as a NL/Lux project it deviates from the dsmr standard. Also the DSMR standard is versioned, that is only partially taken into consideration.

[afarago]

Connected to #2393

[probot-esphome[bot]]

Hey there @glmnet, @zuidwijk, mind taking a look at this issue as it has been labeled with an integration (dsmr) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[afarago]

In the meantime made quite a nice progress solving a teh specific issue. A meter in Hungary (/HLY5/D545-METER) is emitting quite error prone DSMR data: wring unit of measure capitalization, extra spaces, lack of unit of measurement, and an aggregation field at the end summarizing the monthly data 0-0:98.1.0)

The code behind the dsmr is both genius and seems quite bad from coding standards perspective. I was able to understand, debug and fix the code to enable the specific meter to run.

Problematic telegram

Problematic dsmr telegram attached below: szlaci_payload_full.txt szlaci_payload_onewproblem.txt

Fun fact: my solution was to pick a local D1 Mini, and enable USB Serial UART, and feed the data via putty as far as I can get some debugging and fixing done. Would be nice to have a serial loopback (Similar to Stream Server #660 but for TCP input/output without a real UART behind)

Committed local fix

Local solution was to modify both the fields.h and the parser.h files. https://github.com/afarago/dsmr

YAML for D545 meter https://github.com/afarago/dsmr/blob/main/specific_versions/slimmelezer_szlaci_cleaned.yaml

Open issue

Would be committed to contribute/refactor the dsmr component, still my expertise on the issue alone is not sufficient. Would need some guidance and starting direction.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mmakaay]

Oh, I saw this issue for the first time now, because of the staleness message. I'm working on a new component that does not depend on the external parser to be able to more quickly fix issues like this. Thanks for the examples, those are good test inputs. Almost like worst-case scenario tests ;-)