dlms smartmeter support thread

SimonFischer04 commented 6 months ago

Hi, I recently started creating an esphome component for reading data from dlms encryped smartmeters. (used a lot in Austria)

Great thanks to DomiStyle(https://github.com/DomiStyle/esphome-dlms-meter) and firegore for the basic implementation. But - as custom-components are deprecated now - I implemented it in a proper esphome-component, added support for evn smartmeters and did some more smaller fixes / changes.

Requirements esp(32 or 8266) and some mbus to uart converter. More details see for example: https://github.com/DomiStyle/esphome-dlms-meter?tab=readme-ov-file#requirements

Here is a basic yaml config you can try out for now:

If you have Netz NÖ / EVN:

# Enable logging
logger:
  # level: VERBOSE
  # tx_buffer_size: 1024

external_components:
  - source: github://SimonFischer04/esphome@dlms-meter
    components: [ dlms_meter ]

uart:
  rx_pin: GPIO4 # change this to the pin where you connected the mbus adapter board
  baud_rate: 2400
  rx_buffer_size: 1024 # Needed to receive the large packets send by the smart meter

dlms_meter:
  decryption_key: "36C66639E48A8CA4D6BC8B282A793BBB" # change this to your decryption key!
  provider: evn # (optional) key - only set if using evn

sensor:
  - platform: dlms_meter
    voltage_l1:
      name: "Voltage L1"
    voltage_l2:
      name: "Voltage L2"
    voltage_l3:
      name: "Voltage L3"
    current_l1:
      name: "Current L1"
    current_l2:
      name: "Current L2"
    current_l3:
      name: "Current L3"
    active_power_plus:
      name: "Active power taken from grid"
    active_power_minus:
      name: "Active power put into grid"
    active_energy_plus:
      name: "Active energy taken from grid"
    active_energy_minus:
      name: "Active energy put into grid"
    # EVN
    power_factor:
      name: "Power Factor"

text_sensor:
  - platform: dlms_meter
    timestamp:
      name: "timestamp"
    # EVN
    meternumber:
      name: "meterNumber"

any other grid operator:

# Enable logging
logger:
  # level: VERBOSE
  # tx_buffer_size: 1024

external_components:
  - source: github://SimonFischer04/esphome@dlms-meter
    components: [ dlms_meter ]

uart:
  rx_pin: GPIO4 # change this to the pin where you connected the mbus adapter board
  baud_rate: 2400
  rx_buffer_size: 1024 # Needed to receive the large packets send by the smart meter

dlms_meter:
  decryption_key: "36C66639E48A8CA4D6BC8B282A793BBB" # change this to your decryption key!

sensor:
  - platform: dlms_meter
    voltage_l1:
      name: "Voltage L1"
    voltage_l2:
      name: "Voltage L2"
    voltage_l3:
      name: "Voltage L3"
    current_l1:
      name: "Current L1"
    current_l2:
      name: "Current L2"
    current_l3:
      name: "Current L3"
    active_power_plus:
      name: "Active power taken from grid"
    active_power_minus:
      name: "Active power put into grid"
    active_energy_plus:
      name: "Active energy taken from grid"
    active_energy_minus:
      name: "Active energy put into grid"
    reactive_energy_plus:
      name: "Reactive energy taken from grid"
    reactive_energy_minus:
      name: "Reactive energy put into grid"

text_sensor:
  - platform: dlms_meter
    timestamp:
      name: "timestamp"

Would love to get some feedback if it works for you also / you encounter some problems. If you test please report back your findings here:

time when you compiled / installed it (as code in github branch can change if I need to fix something, ...)
works: yes/no
- if its does not work: any error in log? (use log-level: VERBOSE)
which provider (and smartmeter) you have
if you are willing to: a few sample packet-data and your decryption key . i'm planning to add automated tests for this component in the future. But for this I would need some data to test / verify. (or if - like evn - your provider has an example with data and key => please also let me know)
which esp board you are using (8266 or 32)
mbus to uart adapter used (shouldn't matter, but maybe its nice so we can collect a list of some)
... (anything else that might be interesting)

At least for now, this issue should be seen also as a (general) discussion thread around esp(home) / reading dlms encrypted data from smartmeters. So any input / feedback is welcome!

To help with debugging, please provide logs after enabling logger: level: VERBOSE (and tx_buffer_size: 1024 - required to log full packets) PS: be careful with VERY_VERBOSE, as this also logs the decryption key (like the logs also warn at the very beginning)!

PS: my plan is to ultimately get this upstreamed into main esphome repo itself, but want to get some feedback / testing first.

Links to other issues I found around this topic:

ximex commented 6 months ago

will try your impl in the next days/weeks. I have a Kaifa MA309M maybe better name it Netz NÖ instead of EVN? One is the grid operator the other one the energy provider

SimonFischer04 commented 6 months ago

will try your impl in the next days/weeks. I have a Kaifa MA309M

no hurry, thanks

maybe better name it Netz NÖ instead of EVN? One is the grid operator the other one the energy provider

yeah naming things is hard ... 🤣. EVN/ Netz NÖ are quite linked together. Other discussions I found also usually talked about "evn" / "Netz NÖ". synonymously. How would call the key? I definitely don't put a space nor "Ö" in there. "netznoe" maybe?

But let's start with checking if it works. I think a working implementation is more important than some naming detail :).

ximex commented 6 months ago

yes not the most important thing. i would call it netznoe

ximex commented 6 months ago

maybe this overview helps: https://cms.shrdzm.com/wp-content/uploads/Smartmeter-scaled.jpg

SimonFischer04 commented 6 months ago

maybe this overview helps: https://cms.shrdzm.com/wp-content/uploads/Smartmeter-scaled.jpg

I already know this chart, but how should this help? This implementation is only intended for the ones providing an m-bus interface (or identical data format / structure provided with other converters to the the esp uart).

If you mean grid operator vs energy provider, yes I know the difference. (I don't actually have EVN as energy provider here, but netz nö as grid operator). Don't really know why I called it like this ... 😄

ximex commented 6 months ago

I only added as an overview for all the possible providers/devices that are there in austria

jenskn83 commented 6 months ago

Hi Simon, just gave it a try. But i probably did something wrong :-)

Kaifa MA309 VKW

see the attached yaml esphome-web-a2ded9.yaml.txt

And here the result:

` INFO ESPHome 2024.4.1 INFO Reading configuration /config/esphome/esphome-web-a2ded9.yaml... INFO Generating C++ source... INFO Compiling app... Processing smartmeter (board: esp01_1m; framework: arduino; platform: platformio/espressif8266@4.2.1)

HARDWARE: ESP8266 80MHz, 80KB RAM, 1MB Flash Dependency Graph |-- ESPAsyncTCP-esphome @ 2.0.0 |-- ESPAsyncWebServer-esphome @ 3.1.0 |-- DNSServer @ 1.1.1 |-- ESP8266WiFi @ 1.0 |-- ESP8266mDNS @ 1.2 |-- noise-c @ 0.1.4 Compiling .pioenvs/smartmeter/src/esphome-dlms-meter/espdm.cpp.o Compiling .pioenvs/smartmeter/src/esphome/components/socket/bsd_sockets_impl.cpp.o Compiling .pioenvs/smartmeter/src/esphome/components/socket/lwip_raw_tcp_impl.cpp.o Compiling .pioenvs/smartmeter/src/esphome/components/socket/lwip_sockets_impl.cpp.o Compiling .pioenvs/smartmeter/src/esphome/components/socket/socket.cpp.o Compiling .pioenvs/smartmeter/src/esphome/components/text_sensor/filter.cpp.o Compiling .pioenvs/smartmeter/src/esphome/components/text_sensor/text_sensor.cpp.o In file included from src/esphome-dlms-meter/espdm.cpp:1: src/esphome-dlms-meter/espdm.h:7:20: error: conflicting declaration 'const char TAG' 7 | static const char TAG = "espdm"; | ^~~ In file included from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms_meter.h:29:26: note: previous declaration as 'const char const TAG' 29 | static const char const TAG = "dlms_meter"; | ^~~ In file included from src/esphome-dlms-meter/espdm.cpp:1: src/esphome-dlms-meter/espdm.h:29:34: error: 'mqtt' has not been declared 29 | void enable_mqtt(mqtt::MQTTClientComponent mqtt_client, const char topic); | ^~~~ src/esphome-dlms-meter/espdm.h:29:60: error: expected ',' or '...' before '' token 29 | void enable_mqtt(mqtt::MQTTClientComponent mqtt_client, const char topic); | ^ src/esphome-dlms-meter/espdm.h:66:17: error: 'mqtt' does not name a type 66 | mqtt::MQTTClientComponent mqtt_client = NULL; | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:2: src/esphome-dlms-meter/espdm_mbus.h:5:18: error: redefinition of 'const int MBUS_HEADER_INTRO_LENGTH' 5 | static const int MBUS_HEADER_INTRO_LENGTH = 4; // Header length for the intro (0x68, length, length, 0x68) | ^~~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:10, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/mbus.h:6:18: note: 'const int MBUS_HEADER_INTRO_LENGTH' previously defined here 6 | static const int MBUS_HEADER_INTRO_LENGTH = 4; // Header length for the intro (0x68, length, length, 0x68) | ^~~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:2: src/esphome-dlms-meter/espdm_mbus.h:6:18: error: redefinition of 'const int MBUS_FULL_HEADER_LENGTH' 6 | static const int MBUS_FULL_HEADER_LENGTH = 9; // Total header length | ^~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:10, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/mbus.h:7:18: note: 'const int MBUS_FULL_HEADER_LENGTH' previously defined here 7 | static const int MBUS_FULL_HEADER_LENGTH = 9; // Total header length | ^~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:2: src/esphome-dlms-meter/espdm_mbus.h:7:18: error: redefinition of 'const int MBUS_FOOTER_LENGTH' 7 | static const int MBUS_FOOTER_LENGTH = 2; // Footer after frame | ^~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:10, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/mbus.h:8:18: note: 'const int MBUS_FOOTER_LENGTH' previously defined here 8 | static const int MBUS_FOOTER_LENGTH = 2; // Footer after frame | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:2: src/esphome-dlms-meter/espdm_mbus.h:9:18: error: redefinition of 'const int MBUS_MAX_FRAME_LENGTH' 9 | static const int MBUS_MAX_FRAME_LENGTH = 250; // Maximum size of frame | ^~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:10, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/mbus.h:10:18: note: 'const int MBUS_MAX_FRAME_LENGTH' previously defined here 10 | static const int MBUS_MAX_FRAME_LENGTH = 250; // Maximum size of frame | ^~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:2: src/esphome-dlms-meter/espdm_mbus.h:11:18: error: redefinition of 'const int MBUS_START1_OFFSET' 11 | static const int MBUS_START1_OFFSET = 0; // Offset of first start byte | ^~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:10, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/mbus.h:12:18: note: 'const int MBUS_START1_OFFSET' previously defined here 12 | static const int MBUS_START1_OFFSET = 0; // Offset of first start byte | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:2: src/esphome-dlms-meter/espdm_mbus.h:12:18: error: redefinition of 'const int MBUS_LENGTH1_OFFSET' 12 | static const int MBUS_LENGTH1_OFFSET = 1; // Offset of first length byte | ^~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:10, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/mbus.h:13:18: note: 'const int MBUS_LENGTH1_OFFSET' previously defined here 13 | static const int MBUS_LENGTH1_OFFSET = 1; // Offset of first length byte | ^~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:2: src/esphome-dlms-meter/espdm_mbus.h:13:18: error: redefinition of 'const int MBUS_LENGTH2_OFFSET' 13 | static const int MBUS_LENGTH2_OFFSET = 2; // Offset of (duplicated) second length byte | ^~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:10, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/mbus.h:14:18: note: 'const int MBUS_LENGTH2_OFFSET' previously defined here 14 | static const int MBUS_LENGTH2_OFFSET = 2; // Offset of (duplicated) second length byte | ^~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:2: src/esphome-dlms-meter/espdm_mbus.h:14:18: error: redefinition of 'const int MBUS_START2_OFFSET' 14 | static const int MBUS_START2_OFFSET = 3; // Offset of (duplicated) second start byte | ^~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:10, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/mbus.h:15:18: note: 'const int MBUS_START2_OFFSET' previously defined here 15 | static const int MBUS_START2_OFFSET = 3; // Offset of (duplicated) second start byte | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:5:18: error: redefinition of 'const int DLMS_HEADER_LENGTH' 5 | static const int DLMS_HEADER_LENGTH = 16; // Length of the header (total message length <= 127) | ^~~~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:6:18: note: 'const int DLMS_HEADER_LENGTH' previously defined here 6 | static const int DLMS_HEADER_LENGTH = 16; // Length of the header (total message length <= 127) | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:6:18: error: redefinition of 'const int DLMS_HEADER_EXT_OFFSET' 6 | static const int DLMS_HEADER_EXT_OFFSET = 2; // Length to offset when header is extended length (total message length > 127) | ^~~~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:7:18: note: 'const int DLMS_HEADER_EXT_OFFSET' previously defined here 7 | static const int DLMS_HEADER_EXT_OFFSET = 2; // Length to offset when header is extended length (total message length > 127) | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:8:18: error: redefinition of 'const int DLMS_CIPHER_OFFSET' 8 | static const int DLMS_CIPHER_OFFSET = 0; // Offset at which used cipher suite is stored | ^~~~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:9:18: note: 'const int DLMS_CIPHER_OFFSET' previously defined here 9 | static const int DLMS_CIPHER_OFFSET = 0; // Offset at which used cipher suite is stored | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:9:18: error: redefinition of 'const int DLMS_SYST_OFFSET' 9 | static const int DLMS_SYST_OFFSET = 1; // Offset at which length of system title is stored | ^~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:10:18: note: 'const int DLMS_SYST_OFFSET' previously defined here 10 | static const int DLMS_SYST_OFFSET = 1; // Offset at which length of system title is stored | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:11:18: error: redefinition of 'const int DLMS_LENGTH_OFFSET' 11 | static const int DLMS_LENGTH_OFFSET = 10; // Offset at which message length is stored | ^~~~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:12:18: note: 'const int DLMS_LENGTH_OFFSET' previously defined here 12 | static const int DLMS_LENGTH_OFFSET = 10; // Offset at which message length is stored | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:12:18: error: redefinition of 'const int DLMS_LENGTH_CORRECTION' 12 | static const int DLMS_LENGTH_CORRECTION = 5; // Part of the header is included in the DLMS length field and needs to be removed | ^~~~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:13:18: note: 'const int DLMS_LENGTH_CORRECTION' previously defined here 13 | static const int DLMS_LENGTH_CORRECTION = 5; // Part of the header is included in the DLMS length field and needs to be removed | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:16:18: error: redefinition of 'const int DLMS_SECBYTE_OFFSET' 16 | static const int DLMS_SECBYTE_OFFSET = 11; // Offset of the security byte | ^~~~~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:17:18: note: 'const int DLMS_SECBYTE_OFFSET' previously defined here 17 | static const int DLMS_SECBYTE_OFFSET = 11; // Offset of the security byte | ^~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:18:18: error: redefinition of 'const int DLMS_FRAMECOUNTER_OFFSET' 18 | static const int DLMS_FRAMECOUNTER_OFFSET = 12; // Offset of the frame counter | ^~~~~~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:19:18: note: 'const int DLMS_FRAMECOUNTER_OFFSET' previously defined here 19 | static const int DLMS_FRAMECOUNTER_OFFSET = 12; // Offset of the frame counter | ^~~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:19:18: error: redefinition of 'const int DLMS_FRAMECOUNTER_LENGTH' 19 | static const int DLMS_FRAMECOUNTER_LENGTH = 4; // Length of the frame counter (always 4) | ^~~~~~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:20:18: note: 'const int DLMS_FRAMECOUNTER_LENGTH' previously defined here 20 | static const int DLMS_FRAMECOUNTER_LENGTH = 4; // Length of the frame counter (always 4) | ^~~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:3: src/esphome-dlms-meter/espdm_dlms.h:21:18: error: redefinition of 'const int DLMS_PAYLOAD_OFFSET' 21 | static const int DLMS_PAYLOAD_OFFSET = 16; // Offset at which the encrypted payload begins | ^~~~~~~ In file included from src/esphome.h:17, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/dlms.h:22:18: note: 'const int DLMS_PAYLOAD_OFFSET' previously defined here 22 | static const int DLMS_PAYLOAD_OFFSET = 16; // Offset at which the encrypted payload begins | ^~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:5:6: error: multiple definition of 'enum DataType' 5 | enum DataType | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:6:6: note: previous definition here 6 | enum DataType | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:34:6: error: multiple definition of 'enum Medium' 34 | enum Medium | ^~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:35:6: note: previous definition here 35 | enum Medium | ^~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:43:6: error: multiple definition of 'enum CodeType' 43 | enum CodeType | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:44:6: note: previous definition here 44 | enum CodeType | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:63:6: error: multiple definition of 'enum Accuracy' 63 | enum Accuracy | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:66:6: note: previous definition here 66 | enum Accuracy | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:73:18: error: redefinition of 'const int DECODER_START_OFFSET' 73 | static const int DECODER_START_OFFSET = 20; // Offset for start of OBIS decoding, skip header, timestamp and break block | ^~~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:76:18: note: 'const int DECODER_START_OFFSET' previously defined here 76 | static const int DECODER_START_OFFSET = 20; // Offset for start of OBIS decoding, skip header, timestamp and break block | ^~~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:75:18: error: redefinition of 'const int OBIS_TYPE_OFFSET' 75 | static const int OBIS_TYPE_OFFSET = 0; | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:78:18: note: 'const int OBIS_TYPE_OFFSET' previously defined here 78 | static const int OBIS_TYPE_OFFSET = 0; | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:76:18: error: redefinition of 'const int OBIS_LENGTH_OFFSET' 76 | static const int OBIS_LENGTH_OFFSET = 1; | ^~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:79:18: note: 'const int OBIS_LENGTH_OFFSET' previously defined here 79 | static const int OBIS_LENGTH_OFFSET = 1; | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:78:18: error: redefinition of 'const int OBIS_CODE_OFFSET' 78 | static const int OBIS_CODE_OFFSET = 2; | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:81:18: note: 'const int OBIS_CODE_OFFSET' previously defined here 81 | static const int OBIS_CODE_OFFSET = 2; | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:80:18: error: redefinition of 'const int OBIS_A' 80 | static const int OBIS_A = 0; | ^~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:83:18: note: 'const int OBIS_A' previously defined here 83 | static const int OBIS_A = 0; | ^~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:81:18: error: redefinition of 'const int OBIS_B' 81 | static const int OBIS_B = 1; | ^~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:84:18: note: 'const int OBIS_B' previously defined here 84 | static const int OBIS_B = 1; | ^~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:82:18: error: redefinition of 'const int OBIS_C' 82 | static const int OBIS_C = 2; | ^~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:85:18: note: 'const int OBIS_C' previously defined here 85 | static const int OBIS_C = 2; | ^~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:83:18: error: redefinition of 'const int OBIS_D' 83 | static const int OBIS_D = 3; | ^~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:86:18: note: 'const int OBIS_D' previously defined here 86 | static const int OBIS_D = 3; | ^~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:84:18: error: redefinition of 'const int OBIS_E' 84 | static const int OBIS_E = 4; | ^~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:87:18: note: 'const int OBIS_E' previously defined here 87 | static const int OBIS_E = 4; | ^~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:85:18: error: redefinition of 'const int OBIS_F' 85 | static const int OBIS_F = 5; | ^~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:88:18: note: 'const int OBIS_F' previously defined here 88 | static const int OBIS_F = 5; | ^~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:91:16: error: conflicting declaration 'uint8_t ESPDM_TIMESTAMP []' 91 | static uint8_t ESPDM_TIMESTAMP[] | ^~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:94:22: note: previous declaration as 'const uint8_t ESPDM_TIMESTAMP [2]' 94 | static const uint8_t ESPDM_TIMESTAMP[] | ^~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:96:22: error: redefinition of 'const uint8_t ESPDM_SERIAL_NUMBER []' 96 | static const uint8_t ESPDM_SERIAL_NUMBER[] | ^~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:99:22: note: 'const uint8_t ESPDM_SERIAL_NUMBER [2]' previously defined here 99 | static const uint8_t ESPDM_SERIAL_NUMBER[] | ^~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:101:22: error: redefinition of 'const uint8_t ESPDM_DEVICE_NAME []' 101 | static const uint8_t ESPDM_DEVICE_NAME[] | ^~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:104:22: note: 'const uint8_t ESPDM_DEVICE_NAME [2]' previously defined here 104 | static const uint8_t ESPDM_DEVICE_NAME[] | ^~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:110:16: error: conflicting declaration 'uint8_t ESPDM_VOLTAGE_L1 []' 110 | static uint8_t ESPDM_VOLTAGE_L1[] | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:113:22: note: previous declaration as 'const uint8_t ESPDM_VOLTAGE_L1 [2]' 113 | static const uint8_t ESPDM_VOLTAGE_L1[] | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:115:22: error: redefinition of 'const uint8_t ESPDM_VOLTAGE_L2 []' 115 | static const uint8_t ESPDM_VOLTAGE_L2[] | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:118:22: note: 'const uint8_t ESPDM_VOLTAGE_L2 [2]' previously defined here 118 | static const uint8_t ESPDM_VOLTAGE_L2[] | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:120:22: error: redefinition of 'const uint8_t ESPDM_VOLTAGE_L3 []' 120 | static const uint8_t ESPDM_VOLTAGE_L3[] | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:123:22: note: 'const uint8_t ESPDM_VOLTAGE_L3 [2]' previously defined here 123 | static const uint8_t ESPDM_VOLTAGE_L3[] | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:129:22: error: redefinition of 'const uint8_t ESPDM_CURRENT_L1 []' 129 | static const uint8_t ESPDM_CURRENT_L1[] | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:132:22: note: 'const uint8_t ESPDM_CURRENT_L1 [2]' previously defined here 132 | static const uint8_t ESPDM_CURRENT_L1[] | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:134:22: error: redefinition of 'const uint8_t ESPDM_CURRENT_L2 []' 134 | static const uint8_t ESPDM_CURRENT_L2[] | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:137:22: note: 'const uint8_t ESPDM_CURRENT_L2 [2]' previously defined here 137 | static const uint8_t ESPDM_CURRENT_L2[] | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:139:22: error: redefinition of 'const uint8_t ESPDM_CURRENT_L3 []' 139 | static const uint8_t ESPDM_CURRENT_L3[] | ^~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:142:22: note: 'const uint8_t ESPDM_CURRENT_L3 [2]' previously defined here 142 | static const uint8_t ESPDM_CURRENT_L3[] | ^~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:148:22: error: redefinition of 'const uint8_t ESPDM_ACTIVE_POWER_PLUS []' 148 | static const uint8_t ESPDM_ACTIVE_POWER_PLUS[] | ^~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:151:22: note: 'const uint8_t ESPDM_ACTIVE_POWER_PLUS [2]' previously defined here 151 | static const uint8_t ESPDM_ACTIVE_POWER_PLUS[] | ^~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:153:22: error: redefinition of 'const uint8_t ESPDM_ACTIVE_POWER_MINUS []' 153 | static const uint8_t ESPDM_ACTIVE_POWER_MINUS[] | ^~~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:156:22: note: 'const uint8_t ESPDM_ACTIVE_POWER_MINUS [2]' previously defined here 156 | static const uint8_t ESPDM_ACTIVE_POWER_MINUS[] | ^~~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:162:22: error: redefinition of 'const uint8_t ESPDM_ACTIVE_ENERGY_PLUS []' 162 | static const uint8_t ESPDM_ACTIVE_ENERGY_PLUS[] | ^~~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:165:22: note: 'const uint8_t ESPDM_ACTIVE_ENERGY_PLUS [2]' previously defined here 165 | static const uint8_t ESPDM_ACTIVE_ENERGY_PLUS[] | ^~~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:166:22: error: redefinition of 'const uint8_t ESPDM_ACTIVE_ENERGY_MINUS []' 166 | static const uint8_t ESPDM_ACTIVE_ENERGY_MINUS[] | ^~~~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:169:22: note: 'const uint8_t ESPDM_ACTIVE_ENERGY_MINUS [2]' previously defined here 169 | static const uint8_t ESPDM_ACTIVE_ENERGY_MINUS[] | ^~~~~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:175:22: error: redefinition of 'const uint8_t ESPDM_REACTIVE_ENERGY_PLUS []' 175 | static const uint8_t ESPDM_REACTIVE_ENERGY_PLUS[] | ^~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:178:22: note: 'const uint8_t ESPDM_REACTIVE_ENERGY_PLUS [2]' previously defined here 178 | static const uint8_t ESPDM_REACTIVE_ENERGY_PLUS[] | ^~~~~~ In file included from src/esphome-dlms-meter/espdm.cpp:4: src/esphome-dlms-meter/espdm_obis.h:179:22: error: redefinition of 'const uint8_t ESPDM_REACTIVE_ENERGY_MINUS []' 179 | static const uint8_t ESPDM_REACTIVE_ENERGY_MINUS[] | ^~~~~~~ In file included from src/esphome/components/dlms_meter/dlms_meter.h:12, from src/esphome.h:18, from src/esphome-dlms-meter/espdm.h:1, from src/esphome-dlms-meter/espdm.cpp:1: src/esphome/components/dlms_meter/obis.h:182:22: note: 'const uint8_t ESPDM_REACTIVE_ENERGY_MINUS [2]' previously defined here 182 | static const uint8_t ESPDM_REACTIVE_ENERGY_MINUS[] | ^~~~~~~ src/esphome-dlms-meter/espdm.cpp: In member function 'virtual void esphome::espdm::DlmsMeter::loop()': src/esphome-dlms-meter/espdm.cpp:411:26: error: 'class esphome::espdm::DlmsMeter' has no member named 'mqtt_client' 411 | if(this->mqtt_client != NULL) | ^~~ src/esphome-dlms-meter/espdm.cpp:413:27: error: 'class esphome::espdm::DlmsMeter' has no member named 'mqtt_client' 413 | this->mqtt_client->publish_json(this->topic, [=](JsonObject root) | ^~~ src/esphome-dlms-meter/espdm.cpp:413:70: error: 'JsonObject' has not been declared 413 | this->mqtt_client->publish_json(this->topic, [=](JsonObject root) | ^~~~~~ src/esphome-dlms-meter/espdm.cpp: In lambda function: src/esphome-dlms-meter/espdm.cpp:417:48: error: assignment of read-only location '"voltage_l1"[root]' 417 | root["voltage_l1"] = this->voltage_l1->state; | ~~~~~~~^~~~~~~~~ src/esphome-dlms-meter/espdm.cpp:418:48: error: assignment of read-only location '"voltage_l2"[root]' 418 | root["voltage_l2"] = this->voltage_l2->state; | ~~~~~~~^~~~~~~~~ src/esphome-dlms-meter/espdm.cpp:419:48: error: assignment of read-only location '"voltage_l3"[root]' 419 | root["voltage_l3"] = this->voltage_l3->state; | ~~~~~~~^~~~~~~~~ src/esphome-dlms-meter/espdm.cpp:424:48: error: assignment of read-only location '"current_l1"[root]' 424 | root["current_l1"] = this->current_l1->state; | ~~~~~~~^~~~~~~~~ src/esphome-dlms-meter/espdm.cpp:425:48: error: assignment of read-only location '"current_l2"[root]' 425 | root["current_l2"] = this->current_l2->state; | ~~~~~~~^~~~~~~~~ src/esphome-dlms-meter/espdm.cpp:426:48: error: assignment of read-only location '"current_l3"[root]' 426 | root["current_l3"] = this->current_l3->state; | ~~~~~~~^~~~~~~~~ src/esphome-dlms-meter/espdm.cpp:431:55: error: assignment of read-only location '"active_power_plus"[root]' 431 | root["active_power_plus"] = this->active_power_plus->state; | ~~~~~~^~~~~~~~ src/esphome-dlms-meter/espdm.cpp:432:56: error: assignment of read-only location '"active_power_minus"[root]' 432 | root["active_power_minus"] = this->active_power_minus->state; | ~~~~~^~~~~~~ src/esphome-dlms-meter/espdm.cpp:437:56: error: assignment of read-only location '"active_energy_plus"[root]' 437 | root["active_energy_plus"] = this->active_energy_plus->state; | ~~~~~^~~~~~~ src/esphome-dlms-meter/espdm.cpp:438:57: error: assignment of read-only location '"active_energy_minus"[root]' 438 | root["active_energy_minus"] = this->active_energy_minus->state; | ~~~~~~^~~~~~~~ src/esphome-dlms-meter/espdm.cpp:443:58: error: assignment of read-only location '"reactive_energy_plus"[root]' 443 | root["reactive_energy_plus"] = this->reactive_energy_plus->state; | ~~~~~~~~~^~~~~~~~~~~ src/esphome-dlms-meter/espdm.cpp:444:59: error: assignment of read-only location '"reactive_energy_minus"[root]' 444 | root["reactive_energy_minus"] = this->reactive_energy_minus->state; | ~~~~~~^~~~~~~~ src/esphome-dlms-meter/espdm.cpp:449:47: error: assignment of read-only location '"timestamp"[root]' 449 | root["timestamp"] = this->timestamp->state; | ~~~~^~~~~~ src/esphome-dlms-meter/espdm.cpp: At global scope: src/esphome-dlms-meter/espdm.cpp:514:14: error: variable or field 'enable_mqtt' declared void 514 | void DlmsMeter::enable_mqtt(mqtt::MQTTClientComponent mqtt_client, const char topic) | ^~~~~ src/esphome-dlms-meter/espdm.cpp:514:37: error: 'mqtt' has not been declared 514 | void DlmsMeter::enable_mqtt(mqtt::MQTTClientComponent mqtt_client, const char topic) | ^~~~ src/esphome-dlms-meter/espdm.cpp:514:64: error: 'mqtt_client' was not declared in this scope 514 | void DlmsMeter::enable_mqtt(mqtt::MQTTClientComponent mqtt_client, const char topic) | ^~~ src/esphome-dlms-meter/espdm.cpp:514:77: error: expected primary-expression before 'const' 514 | void DlmsMeter::enable_mqtt(mqtt::MQTTClientComponent mqtt_client, const char topic) | ^~~~~ Compiling .pioenvs/smartmeter/src/esphome/components/uart/uart.cpp.o *** [.pioenvs/smartmeter/src/esphome-dlms-meter/espdm.cpp.o] Error 1 ========================= [FAILED] Took 15.38 seconds =========================`

SimonFischer04 commented 6 months ago

@jenskn83 seems like it complains something about mqtt. Neither the component itself nor your attached yaml contains anything mqtt related.

Did you try / use the DomiStyle/esphome-dlms-meter implementation before? - seems to cause conflicts here. If yes and possible, please remove the other custom component folder before trying again.

Furthermore: because you have vkw you probably have to comment out (put # at beginning) The "provider: evn" and the evn related sensors

ximex commented 6 months ago

@SimonFischer04 i got it some minutes before to work. i needed some time to figure out i flipped TX and RX on the ESP32. the only thing is that my smart meter doesn't support "Reactive energy" 218_13_SmartMeter_Kundenschnittstelle_2604_web.pdf

SimonFischer04 commented 6 months ago

@ximex means you got it working?

And about the "Reactive Energy": the provided yaml above is just a "maximum configuration" you can just remove the sensor and everything should still just work.

ximex commented 6 months ago

yes works now. but i have some strange spikes (ignore that the sensor is name BME280 - reused it for this test) grafik grafik

SimonFischer04 commented 6 months ago

@ximex great to here that you got it working.

I find it hard to believe that my impl could somehow create these spikes. Reading through other threads seems that this could be a bug from the smartmeter sometimes sending bogus data. The usual workaround is f.e. to apply a filter to tge sensors and use median over multiple values. See here: https://www.esphome.io/components/sensor/

Did you use some other impl before to read data from your smartmeter before and did not experience this issue?

ximex commented 6 months ago

I tested the last days with https://cms.shrdzm.com/produkt/smartmeter-modul/ There i didn't see such a spike. but there is a mqtt server in between and don't know what this device all does.

i got now another spike but i have now the logs open. See "Active power taken from grid":

[22:04:21][D][uart_debug:114]: <<< 68:FA:FA:68:53:FF:00:01:67:DB:08:4B:46:4D:65:50:9B:C7:13:81:F8:20:00:03:CF:55:2C:9B:01:A4:2A:B0:B3:B9:1E:63:EC:10:13:B6:D2:82:42:88:40:6C:B3:33:08:28:EE:EC:61:31:8B:63:6F:B7:32:61:CD:2D:0E:E1:00:B0:30:08:98:C4:96:E1:6B:EC:B7:1A:64:41:03:C8:E1:55:63:E6:EC:4C:55:B8:AA:2D:18:F0:5C:10:B0:D0:E8:90:D3:1C:E4:C4:23:68:22:50:0C:97:1A:D6:97:F1:08:68:26:CA:F3:1D:34:9B
[22:04:21][D][uart_debug:114]: <<< 90:F7:5D:42:FC:8A:29:7A:6F:97:33:9A:94:DD:55:84:BA:38:A7:C4:E3:CB:46:26:B5:55:BB:EA:F7:D4:78:D6:9D:EF:D4:53:A1:07:50:C9:A8:93:ED:7F:F0:9F:EE:EF:84:9B:48:9D:BD:AD:7C:95:B2:FC:BB:8C:06:DB:6A:D7:31:45:A5:6D:CF:BE:5B:EB:1A:99:D4:C0:3D:A3:9D:EF:FC:E6:72:A1:88:C2:B1:EF:D9:C6:CA:5E:DE:54:84:EC:E1:4B:7C:F7:F3:60:BB:80:99:9C:DE:FA:48:69:37:0E:ED:48:3C:E3:EC:2A:2A:9A:95:9A:4A:34:B8:03:B4:9B:C3:36:13:9B:FC:9A:70:16
[22:04:22][D][uart_debug:114]: <<< 68:14:14:68:53:FF:11:01:67:FF:20:71:E0:C6:B8:8C:72:89:F5:75:42:A3:B1:73:B3:16
[22:04:23][V][dlms_meter:479]: 68.FA.FA.68.53.FF.00.01.67.DB.08.4B.46.4D.65.50.9B.C7.13.81.F8.20.00.03.CF.55.2C.9B.01.A4.2A.B0.B3.B9.1E.63.EC.10.13.B6.D2.82.42.88.40.6C.B3.33.08.28.EE.EC.61.31.8B.63.6F.B7.32.61.CD.2D.0E.E1.00.B0.30.08.98.C4.96.E1.6B.EC.B7.1A.64.41.03.C8.E1.55.63.E6.EC.4C.55.B8.AA.2D.18.F0.5C.10.B0.D0.E8.90.D3.1C.E4.C4.23.68.22.50.0C.97.1A.D6.97.F1.08.68.26.CA.F3.1D.34.9B.90.F7.5D.42.FC.8A.29.7A.6F.97.33.9A.94.DD.55.84.BA.38.A7.C4.E3.CB.46.26.B5.55.BB.EA.F7.D4.78.D6.9D.EF.D4.53.A1.07.50.C9.A8.93.ED.7F.F0.9F.EE.EF.84.9B.48.9D.BD.AD.7C.95.B2.FC.BB.8C.06.DB.6A.D7.31.45.A5.6D.CF.BE.5B.EB.1A.99.D4.C0.3D.A3.9D.EF.FC.E6.72.A1.88.C2.B1.EF.D9.C6.CA.5E.DE.54.84.EC.E1.4B.7C.F7.F3.60.BB.80.99.9C.DE.FA.48.69.37.0E.ED.48.3C.E3.EC.2A.2A.9A.95.9A.4A.34.B8.03.B4.9B.C3.36.13.9B.FC.9A.70.16.68.14.14.68.53.FF.11.01.67.FF.20.71.E0.C6.B8.8C.72.89.F5.75.42.A3.B1.73.B3.16 (282)
[22:04:23][V][dlms_meter:046]: Parsing M-Bus frames
[22:04:23][V][dlms_meter:052]: MBUS: Parsing frame
[22:04:23][V][dlms_meter:052]: MBUS: Parsing frame
[22:04:23][V][dlms_meter:095]: Parsing DLMS header
[22:04:23][V][dlms_meter:161]: Decrypting payload
[22:04:23][V][dlms_meter:193]: Decrypted payload: 243
[22:04:23][V][dlms_meter:194]: 0F.80.F1.4B.72.0C.07.E8.04.1D.01.16.04.14.00.FF.88.80.02.23.09.0C.07.E8.04.1D.01.16.04.14.00.FF.88.80.09.06.01.00.01.08.00.FF.06.00.3D.F2.56.02.02.0F.00.16.1E.09.06.01.00.02.08.00.FF.06.00.00.00.00.02.02.0F.00.16.1E.09.06.01.00.01.07.00.FF.06.81.00.00.78.02.02.0F.00.16.1B.09.06.01.00.02.07.00.FF.06.00.00.00.00.02.02.0F.00.16.1B.09.06.01.00.20.07.00.FF.12.09.24.02.02.0F.FF.16.23.09.06.01.00.34.07.00.FF.12.09.32.02.02.0F.FF.16.23.09.06.01.00.48.07.00.FF.12.09.1B.02.02.0F.FF.16.23.09.06.01.00.1F.07.00.FF.12.00.03.02.02.0F.FE.16.21.09.06.01.00.33.07.00.FF.12.00.1F.02.02.0F.FE.16.21.09.06.01.00.47.07.00.FF.12.00.34.02.02.0F.FE.16.21.09.06.01.00.0D.07.00.FF.12.03.3C.02.02.0F.FD.16.FF.XX.XX.XX.XX.XX.XX.XX.XX.XX.XX.XX.XX.XX.XX (243)
[22:04:23][V][dlms_meter:203]: Decoding payload
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 7
[22:04:23][V][dlms_meter:248]: currentPosition: 21
[22:04:23][V][dlms_meter:309]: Found Timestamp without obisMedium
[22:04:23][V][dlms_meter:385]: Arrived on OctetString
[22:04:23][V][dlms_meter:386]: currentPosition: 21, plaintext: 12
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 1
[22:04:23][V][dlms_meter:248]: currentPosition: 43
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 1
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 1
[22:04:23][V][dlms_meter:248]: currentPosition: 100
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 1
[22:04:23][V][dlms_meter:248]: currentPosition: 119
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 1
[22:04:23][V][dlms_meter:248]: currentPosition: 136
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 1
[22:04:23][V][dlms_meter:248]: currentPosition: 153
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 1
[22:04:23][V][dlms_meter:248]: currentPosition: 170
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 1
[22:04:23][V][dlms_meter:248]: currentPosition: 187
[22:04:23][V][dlms_meter:248]: currentPosition: 204
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 1
[22:04:23][V][dlms_meter:248]: currentPosition: 221
[22:04:23][V][dlms_meter:247]: obisCode (OBIS_A): 49
[22:04:23][V][dlms_meter:248]: currentPosition: 230
[22:04:23][V][dlms_meter:312]: Found MeterNumber without obisMedium
[22:04:23][V][dlms_meter:385]: Arrived on OctetString
[22:04:23][V][dlms_meter:386]: currentPosition: 230, plaintext: 12
[22:04:23][V][dlms_meter:419]: Constructing MeterNumber...
[22:04:23][I][dlms_meter:459]: Received valid data
[22:04:23][V][sensor:043]: 'Voltage L1': Received new state 234.000000
[22:04:23][D][sensor:094]: 'Voltage L1': Sending state 234.00000 V with 1 decimals of accuracy
[22:04:23][V][sensor:043]: 'Voltage L2': Received new state 235.399994
[22:04:23][D][sensor:094]: 'Voltage L2': Sending state 235.39999 V with 1 decimals of accuracy
[22:04:23][V][sensor:043]: 'Voltage L3': Received new state 233.100006
[22:04:23][D][sensor:094]: 'Voltage L3': Sending state 233.10001 V with 1 decimals of accuracy
[22:04:23][V][sensor:043]: 'Current L1': Received new state 0.030000
[22:04:23][D][sensor:094]: 'Current L1': Sending state 0.03000 A with 2 decimals of accuracy
[22:04:23][V][sensor:043]: 'Current L2': Received new state 0.310000
[22:04:23][D][sensor:094]: 'Current L2': Sending state 0.31000 A with 2 decimals of accuracy
[22:04:23][V][sensor:043]: 'Current L3': Received new state 0.520000
[22:04:23][D][sensor:094]: 'Current L3': Sending state 0.52000 A with 2 decimals of accuracy
[22:04:23][V][sensor:043]: 'Active power taken from grid': Received new state 2164260864.000000
[22:04:23][D][sensor:094]: 'Active power taken from grid': Sending state 2164260864.00000 W with 0 decimals of accuracy
[22:04:23][V][sensor:043]: 'Active power put into grid': Received new state 0.000000
[22:04:23][D][sensor:094]: 'Active power put into grid': Sending state 0.00000 W with 0 decimals of accuracy
[22:04:23][V][sensor:043]: 'Active energy taken from grid': Received new state 4059734.000000
[22:04:23][D][sensor:094]: 'Active energy taken from grid': Sending state 4059734.00000 Wh with 0 decimals of accuracy
[22:04:23][V][sensor:043]: 'Active energy put into grid': Received new state 0.000000
[22:04:23][D][sensor:094]: 'Active energy put into grid': Sending state 0.00000 Wh with 0 decimals of accuracy
[22:04:23][V][sensor:043]: 'Reactive energy taken from grid': Received new state 0.000000
[22:04:23][D][sensor:094]: 'Reactive energy taken from grid': Sending state 0.00000 Wh with 0 decimals of accuracy
[22:04:23][V][sensor:043]: 'Reactive energy put into grid': Received new state 0.000000
[22:04:23][D][sensor:094]: 'Reactive energy put into grid': Sending state 0.00000 Wh with 0 decimals of accuracy
[22:04:23][V][sensor:043]: 'Power Factor': Received new state 0.828000
[22:04:23][D][sensor:094]: 'Power Factor': Sending state 0.82800  with 3 decimals of accuracy
[22:04:23][V][text_sensor:013]: 'timestamp': Received new state 2024-04-29T22:04:20Z
[22:04:23][D][text_sensor:064]: 'timestamp': Sending state '2024-04-29T22:04:20Z'
[22:04:23][V][text_sensor:013]: 'meterNumber': Received new state XXX
[22:04:23][D][text_sensor:064]: 'meterNumber': Sending state 'XXX'
[22:04:23][W][component:237]: Component dlms_meter took a long time for an operation (612 ms).
[22:04:23][W][component:238]: Components should block for at most 30 ms.

masked the meter number

Are this warnings expected?

[22:04:13][W][component:237]: Component dlms_meter took a long time for an operation (606 ms).
[22:04:13][W][component:238]: Components should block for at most 30 ms.

Edit: also got now one with 0V and this:

[22:32:32][E][dlms_meter:078]: MBUS: Invalid stop byte
...
[22:33:47][E][dlms_meter:210]: OBIS: Unsupported OBIS header type: 2f

after removing the debug logs and the reactive energy i'm now down to ~130ms:

[22:34:23][W][component:237]: Component dlms_meter took a long time for an operation (137 ms).
[22:34:23][W][component:238]: Components should block for at most 30 ms.

SimonFischer04 commented 6 months ago

I tested the last days with https://cms.shrdzm.com/produkt/smartmeter-modul/

Do you by chance remember how often this sends data? If there is a greater than 5sec delay its very possible they also do some filtering

Are this warnings expected?

Kind of. Unfortunately I don't see much i can do here as reading the data and decrying takes time and I also dont want to miss a byte while reading. Also sending that much logs takes time for the esp - but verbose logging should be enabled only for debugging anyways.

But should not be a problem, a lot of other components also create such warnings. Just mean other components are blocked while this one is working. (Do webserver f.e. does sometakes take some time to render,...) If your esp is only responsible for reading the smartmeter and sending data to other source for further processing, this really shouldn't matter.

i got now another spike but i have now the logs open. See "Active power taken from grid":

I will (hopefully) look through the decrypted packet you provided a bit more tomorrow to see if this gives some more insight.

OBIS: Unsupported OBIS header type: 2f

Do you by chance have the full decrypted packet for this one also?

jenskn83 commented 6 months ago

@jenskn83 seems like it complains something about mqtt. Neither the component itself nor your attached yaml contains anything mqtt related.

Did you try / use the DomiStyle/esphome-dlms-meter implementation before? - seems to cause conflicts here. If yes and possible, please remove the other custom component folder before trying again.

Furthermore: because you have vkw you probably have to comment out (put # at beginning) The "provider: evn" and the evn related sensors

Thanks for your answer. i tried with a new setup.

now i receive a different error message:

[09:40:43][E][dlms_meter:072]: MBUS: Frame too big for received data [09:40:44][E][dlms_meter:072]: MBUS: Frame too big for received data [09:40:48][E][dlms_meter:072]: MBUS: Frame too big for received data [09:40:49][E][dlms_meter:072]: MBUS: Frame too big for received data [09:40:53][D][api:102]: Accepted 192.168.178.70 [09:40:53][W][api.connection:091]: : Socket operation failed: HANDSHAKESTATE_READ_FAILED errno=11 [09:40:53][E][dlms_meter:072]: MBUS: Frame too big for received data [09:40:54][E][dlms_meter:072]: MBUS: Frame too big for received data [09:40:58][E][dlms_meter:072]: MBUS: Frame too big for received data [09:40:59][E][dlms_meter:064]: MBUS: Length bytes do not match

i attached the Log file. (logs_test2_logs.txt) logs_test2_logs.txt

when i switch to logging level "Very_Verbose" the built failed with several error messages (logs_test2_VERY_VERBOSE.txt) logs_test2_VERY_VERBOSE.txt

here the yaml (meter.yaml) meter.yaml.txt

can you help me with these issues please? Jens

SimonFischer04 commented 6 months ago

according to this data it seems unfortunately that the smartmeter is in fact sending wrong data. when checking with https://www.gurux.fi/GuruxDLMSTranslator and using the OBIS code - 1.0.1.7.0.255 Momentanleistung P+ found here: https://www.netz-noe.at/Download-(1)/Smart-Meter/218_9_SmartMeter_Kundenschnittstelle_lektoriert_14.aspx I get an value of 81000078 W as Active power taken from grid'. So not quite 2164260864.000000W - seems like my impl cant handle that big values and overflows - but 81000078 W is still wrong.

maybe related to that? https://github.com/DomiStyle/esphome-dlms-meter/issues/7#issuecomment-1062191356

Reading this seems like they got partly just junk data. Don't think how it could result in (single) packets with such peeks.

i guess adding a filter as workaround is the best option we have so far. see here f.e. (also others discussing / seeing weird values): https://github.com/DomiStyle/esphome-dlms-meter/issues/16#issuecomment-1100924968

SimonFischer04 commented 6 months ago

@jenskn83

[09:40:53][W][api.connection:091]: : Socket operation failed: HANDSHAKESTATE_READ_FAILED errno=11

I assume: you set "baud_rate: 0" in the logger section of your yaml because you use an 8266 which has only 1 uart (therefore you get the logs from the webserver). right?

i attached the Log file. (logs_test2_logs.txt)

this alone does not give much insight unfortunately

when i switch to logging level "Very_Verbose" the built failed with several error messages (logs_test2_VERY_VERBOSE.txt)

very weird that it can't compile basic esphome logging functionality. can you please try with loglevel VERBOSE. PS: be careful with VERY_VERBOSE, as this also logs the decryption key (as the logs also warn you at the very beginning)!

Furthermore: are you able to temporarily test with an esp32? So we better pinpoint the root issue. 8266 should theoretically work, but I'm using an esp32 here and did not hook up the smartmeter and fully test the 8266 version.

jenskn83 commented 6 months ago

@SimonFischer04 thanks a lot. with an esp32 your implementation runs out of the box!

Jens

SimonFischer04 commented 6 months ago

@SimonFischer04 thanks a lot. with an esp32 your implementation runs out of the box!

Jens

Thanks for reporting back and great that it works for you.

I guess I will test the 8266 implementation again myself and see if I can get it working.

SimonFischer04 commented 6 months ago

just updated the initial comment to clarify / provide examples for the generic and netznoe grid operator settings. also added info about logging (levels)

Augar commented 6 months ago

Running a ESP8266 d1_mini EVN KaifaMA309M without any issue :+1:

NECH2004 commented 5 months ago

Hello Simon, the setup ESP32, KaifaMA309M and SalzburgNetz works without any problem.

FYI: rx pin is GPIO17 (UART2) (tx is GPIO 16)

I' ve used following adapter: https://de.aliexpress.com/item/4000153285401.html (slave adapter) similar to: https://www.amazon.de/-/en/Serial-Master-Converter-Communication-Control-TTL-MBUS-Slave/dp/B0BXPTMGDT

mind: use 3.3 V NOT 5 V (Vin) when wireing to ESP32

erik-toth commented 3 months ago

Hey Simon, I tried your version and now everything works as it should! Thanks a lot.

My setup is: ESP32, Kaifa MA309M and my engery provider is TINETZ!

jenskn83 commented 2 months ago

@ximex Hey Thomas,

hast du mittlerweile eine Lösung für die Spikes in den Rohdaten gefunden? Sie hauen mir regelmäßig die Diagramme durcheinander...

Viele Grüße Jens

ximex commented 2 months ago

sorry hab es noch nicht weiter untersucht

SimonFischer04 / esphome

dlms smartmeter support thread #1

` INFO ESPHome 2024.4.1 INFO Reading configuration /config/esphome/esphome-web-a2ded9.yaml... INFO Generating C++ source... INFO Compiling app... Processing smartmeter (board: esp01_1m; framework: arduino; platform: platformio/espressif8266@4.2.1)