search

esphome / feature-requests

ESPHome Feature Request Tracker
https://esphome.io/
411 stars 26 forks source link

SML sensor for reading smartmeters #1041

Closed Syntoxr closed 2 years ago

Syntoxr commented 3 years ago

Describe the problem you have/What new integration you would like

Hi, I would love to see SML (smart message language) support for ESPHome as (I think) many smart meters use the SML language in combination with the D0 or other reading heads. I use a phototransistor like described in the repo linked below of @mruettgers.

Please describe your use case for this integration and alternatives you've tried:

My use case, as described, is to read smart meters without having to buy overpriced hardware from the manufacturer of smart meters. The Alternative I'm currently using is this one: https://github.com/mruettgers/SMLReader I have multiple smart meters and it supports only one input. Also ESPHome integrates better into HomeAssistant and it's obviously easier to manage everything from one place.

Additional context

This site is written in German but I think most of it is translatable and it describes the structure of this protocol well: https://www.msxfaq.de/sonst/bastelbude/smartmeter_d0_sml_protokoll.htm

Thank you for your work 👍

mruettgers commented 3 years ago

Hey Syntoxr,

just to clarify, SMLReader does support more than one input. Because it makes use of software based serial access, you can use almost any GPIO pin to attach further meters.

And at least one has confirmed that it is working :-). https://github.com/strandaster/SMLReader/commit/be11239224410147e271662d3eabd3e53c9c5dd5#commitcomment-42372498

Syntoxr commented 3 years ago

Hi, My fault, I'll try this out but I think it would still be a useful addition for ESP home. Thanks for pointing to this.

farbefreak commented 3 years ago

I am in for it. My supply company will switch to smart meters next year and i would love to interface with it properly.

Hoelli4C commented 3 years ago

SML Support in ESPHome would be awesome. +1 from me ;)

PKM85 commented 3 years ago

Same here! Would very much appreciate SML support...

autox86 commented 3 years ago

This would really very nice to have it in ESPHome. @mruettgers Your project is awesome already, would it be able for you to add the native home assistant api? This is the main advantage of esohome for me

Hoelli4C commented 3 years ago

Since Home Assistant 2021.8 just added the Energy Dashboard, I wanted to bump this. I hope this is ok.

tazginpax-ho commented 3 years ago

I would also like to see this integrated in Esphome rather than my current solution via tasmota ;)

Styx85 commented 3 years ago

There exists a library in C. Maybe ist is of use.

jarny0815 commented 3 years ago

SML Integration in ESPHome would be very cool. Many smartmeters use SML records to send their values. Very cheap sensors (only a phototransistor + pullup resistor + 2 neodym magnets) could easily build (cost less than 1 Euro).

BMOD89 commented 3 years ago

I have played a bit around with esphome and the existing dsmr component. I changed the package so that I succeeded in reading SML packages in. It was relatively easy...

Now I am thinking how the parsing should be done. One option would be to adapt the dsmr parser. I didn't look into it in detail yet if it makes sense.

@mruettgers is using the libSML package from here https://github.com/volkszaehler/libsml In the contribution guide is written "The number of external libraries should be kept to a minimum".

So my question is if this is an option for esphome or not. I am also not clear how the integration into esphome would be possible. What is the recommendation of the esphome maintainers? @OttoWinter could you please help?

BMOD89 commented 3 years ago

I just saw that DSMR is also using a external library in the final implementation in esphome. So it should be fine here as well.

Diddlik commented 3 years ago

I just saw that DSMR is also using a external library in the final implementation in esphome. So it should be fine here as well.

Maybe it would be possible to implemewnt an custom component for parsing? Till now I tried Tasmota, ESPEasy, SMLReader - all of them a very unflexible and not so easy to integrate...

BMOD89 commented 3 years ago

I successfully integrated libsml into my custom component via plattform.io. Parsing the message is working. I can see the values in the logger.
Atm I'm trying to understand how the values need to be defined and how to publish the values. Let's see :)

Diddlik commented 3 years ago

Nice job! I hope it works out for you! Because it would be the best solution so far!

alengwenus commented 3 years ago

I was also playing around with a custom component which reads out the total energy and active power from my smart meter using SML. „Parsing“ is done by just specifying the bytes (position and length) in the SML file. Works fine for me. Although I didn’t put much effort into it, the crc16 check and some logging to display the SML file is implemented. The repo is here. Maybe it helps someone…

Diddlik commented 3 years ago

I was also playing around with a custom component which reads out the total energy and active power from my smart meter using SML. „Parsing“ is done by just specifying the bytes (position and length) in the SML file. Works fine for me. Although I didn’t put much effort into it, the crc16 check and some logging to display the SML file is implemented. The repo is here. Maybe it helps someone…

I looked at your solution, you do not use the sml lib, but only the parser for your counter, ie for another would be necessary to adjust the parser?

alengwenus commented 3 years ago

I looked at your solution, you do not use the sml lib, but only the parser for your counter, ie for another would be necessary to adjust the parser?

Yes, as I said: You have to identify the corresponding byte positions by yourself. I guess a more mature solution would parse for the specified OBIS numbers and then derive the values. I don't know if libsml helps here or how complicated it is to write a minimal solution for parsing.

Diddlik commented 3 years ago

I looked at your solution, you do not use the sml lib, but only the parser for your counter, ie for another would be necessary to adjust the parser?

Yes, as I said: You have to identify the corresponding byte positions by yourself. I guess a more mature solution would parse for the specified OBIS numbers and then derive the values. I don't know if libsml helps here or how complicated it is to write a minimal solution for parsing.

Can you maybe explain how I can do it for my counter. That would be current quick and durty but working solution. I have Iskra 681.

https://wiki.volkszaehler.org/hardware/channels/meters/power/edl-ehz/iskraemeco_mt681

BMOD89 commented 3 years ago

I have uploaded my test to https://github.com/BMOD89/sml. Basically I adapted dsmr so I can read the values.

I have created a "mapping" table which maps the OBIS ID to a name, so I can configure it in a similar way as dsmr. But this solution is not really universal. It seems to me that some meters use different codes for the same type of value. For example @Diddlik's meter uses 1-0:15.7.0 for the power while mine uses 1-0:16.7.0. But for now the current "solution" is working. I agree with @alengwenus make a mapping between the OBIS ID and the name in the yaml file would be much better. I was thinking about that when I started, but I am not sure yet how to realize that. I need to check other components, how this could be done.

Currently I see the follwoing open points:

@Diddlik, you should be able to test it with your meter. I have added a second mapping for the power which is called power_delivered_2. If the table in the volkszaehler wiki is correct, this should work for your device.

Diddlik commented 3 years ago

not bad. I am currently working on the alengwenus' solution, next will test your! But it looks very good!

1individuum commented 3 years ago

I have uploaded my test to https://github.com/BMOD89/sml. Basically I adapted dsmr so I can read the values.

Thank you very mutch. Works fine with my smart meter "Norax3d"

Hedda commented 2 years ago

Do you guys think SML is also being using by Kamstrup smart water meters which feature a similar bidirectional optical IR eye head?

If so please see request https://github.com/esphome/feature-requests/issues/1402

klaus1956 commented 2 years ago

May be it is woth to add my Gas-Meter Idea here too: https://github.com/klaasnicolaas/home-assistant-glow/issues/42

My plan is to work on this in Oct. 21

Bascht74 commented 2 years ago

Great that there is a solution close. I am really excited to test your Version, BMOD89 (might take some time to deliver the ALI-ESPs :-). This would be much more flexible than the EDL21 Integration in HA, e.g. I can publish the values directly to my wallbox (OpenWB) via mqtt.

Sebastian

alengwenus commented 2 years ago

I tried to start over using the TeleInfo component as a template. Finally I gave up and started from scratch. 🙄 Well, now I come around with a more general approach where we can get the values by specifying the OBIS codes (https://github.com/alengwenus/esphome_components). A typical configuration for this solution looks like:

uart:
  id: uart_bus
  tx_pin: GPIO1
  rx_pin: GPIO3
  baud_rate: 9600

sml:
  uart_id: uart_bus
  logging: false

sensor:
 - platform: sml
    name: "Total energy"
    obis: "1-0:1.8.0"
    unit_of_measurement: kWh
    accuracy_decimals: 1
    filters:
      - multiply: 0.0001

text_sensor:
  - platform: sml
    name: "Manufacturer"
    obis: "129-129:199.130.3"
    format: text

For this I wrote a minimal SML parser which is integrated into the component. To get the available OBIS codes there is a HASS event based logging mechanism available. I also spent an hour to write some documentation. Would be great if someone could test and/or give some feedback if this approach is heading into the right direction. The whole stuff is getting more and more complicated, so don't be too hard on me if it's not perfectly mature. But it works quite well for my EMH smart meter.

BMOD89 commented 2 years ago

@alengwenus looks cool and much more flexible than my current approach. I will test it next weekend and give you feedback.

autox86 commented 2 years ago

@alengwenus : First, many thanks for your code and work! I really love it! If you like, let's have a beer on my costs, living close to Lingen, so this would be feasable ;)

You wrote: To get the available OBIS codes there is a HASS event based logging mechanism available.

From your documentation, I can see that I enable logging and can check in HomeAssistant than. Is that for debugging purpose or do I need to have that enabled the whole time thru?

alengwenus commented 2 years ago

From your documentation, I can see that I enable logging and can check in HomeAssistant than. Is that for debugging purpose or do I need to have that enabled the whole time thru?

@cheisig This is only for identifying the OBIS codes sent from your smart meter. You may disable it after you've figured them out.

autox86 commented 2 years ago

Cool thanks a lot. I assume I can use them from volkszaehler as well.

Let me test yours the next days, just as info, I have an easymeter Q3M and use 1.8.0 for in and 2.8.0 for out.

Thanks again!

ChristophGoth commented 2 years ago

I'm using a Easymeter Q3M too and it's working like a charm, thanks @alengwenus

Syntoxr commented 2 years ago

Jep, works like a charm. One suggestion I have is to publish the OBIS codes inside the node's debug log (like the dallas platform does) Many thanks for your work @alengwenus :+1:

alengwenus commented 2 years ago

Jep, works like a charm. One suggestion I have is to publish the OBIS codes inside the node's debug log (like the dallas platform does) Many thanks for your work @alengwenus 👍

Thanks for testing! I just added the logging to the node's debug log. Much better than using the HASS event. Well, looks like I should start preparing a PR.

DenisMir commented 2 years ago

I'd like to add a request for all the german (especially bavarian users) which got a Holley DTZ541 powermeter with buggy SML implementation. It would be awesome if there could be added a flag to allow integrating the following fix from libsml -> https://github.com/volkszaehler/libsml/issues/75 / https://github.com/volkszaehler/libsml/pull/76 which allows parsing the buggy SML from these power meters that got installed in a lot of households in that area. (I got one of these as well and would help testing this out)

alengwenus commented 2 years ago

@DenisMir The parser is not sensitive to the malformed SML_Time field (which is not used anyway). But I added an evasive check for the CRC16/KERMIT checksum if the CRC16/X25 fails. Unfortunately I don't have any test data. So this is a shot in the dark. ;-)

DenisMir commented 2 years ago

@alengwenus Yeah no problem I will test it out when it is ready and get back with my results. (this power meter drives me nuts with its faulty implementation of SML... :-))

BMOD89 commented 2 years ago

I have some Issues with my meter (SGM-C4). It seems to work if I have disabled the extended data set. But when I enable it, the ESP crashes.

I have debugged it a bit. I guess it is because of memory issues, but I am not sure yet. I will post some logs during the next days...

One point which I saw while debugging, I wonder if it should be checked here, if this->record_ is true. I think it could cause problems during a bad timed first start, if the end bytes are detected before a complete message is received, or?

One general question, would it be beneficial, to check a proper formatted OBIS code in the python script, so that the user gets an error message if the obis code is not correct?

BMOD89 commented 2 years ago

Here is a log where I have changed the Inf setting on my smart meter while the ESP was running: 

[19:27:14][D][sml:078]: OBIS info:
[19:27:14][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.50.1  [0x454652]
[19:27:14][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.1.0  [0x0a014546522002ce50e5]
[19:27:14][D][sml:085]:   (0a014546522002ce50e5) 1-0:1.8.0  [0x013cdd85]
[19:27:14][D][sml:085]:   (0a014546522002ce50e5) 1-0:2.8.0  [0x00]
[19:27:14][D][sml:085]:   (0a014546522002ce50e5) 1-0:0.2.0  [0x30332e3030]
[19:27:14][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.90.2  [0x42bd]
[19:27:14][D][sml:085]:   (0a014546522002ce50e5) 1-0:97.97.0  [0x0000]
[19:27:14][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.50.1  [0x50312e322e3132]
[19:27:14][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.50.4  [0x0422]
[19:27:15][D][sml:078]: OBIS info:
[19:27:15][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.50.1  [0x454652]
[19:27:15][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.1.0  [0x0a014546522002ce50e5]
[19:27:15][D][sml:085]:   (0a014546522002ce50e5) 1-0:1.8.0  [0x013cdd87]
[19:27:15][D][sml:085]:   (0a014546522002ce50e5) 1-0:2.8.0  [0x00]
[19:27:15][D][sml:085]:   (0a014546522002ce50e5) 1-0:0.2.0  [0x30332e3030]
[19:27:15][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.90.2  [0x42bd]
[19:27:15][D][sml:085]:   (0a014546522002ce50e5) 1-0:97.97.0  [0x0000]
[19:27:15][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.50.1  [0x50312e322e3132]
[19:27:15][D][sml:085]:   (0a014546522002ce50e5) 1-0:96.50.4  [0x0422]
[19:27:17]Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
[19:27:17]Core 1 register dump:
[19:27:17]PC      : 0x400d6026  PS      : 0x00060630  A0      : 0x800d60f8  A1      : 0x3ffb1cf0  
INFO Need to fetch platformio IDE-data, please stand by
INFO Running:  platformio run -d stromzaehler -t idedata
WARNING Decoded 0x400d6026: std::vector<unsigned char, std::allocator<unsigned char> >::size() const at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/stl_vector.h:655
 (inlined by) std::vector<unsigned char, std::allocator<unsigned char> >::operator=(std::vector<unsigned char, std::allocator<unsigned char> > const&) at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:191
[19:27:18]A2      : 0x3ffb1e54  A3      : 0x0000000c  A4      : 0x00000000  A5      : 0x3ffb1e3c  
[19:27:18]A6      : 0x00000000  A7      : 0x00000000  A8      : 0x800d60b4  A9      : 0x3ffb1cd0  
[19:27:18]A10     : 0x3ffb1cf4  A11     : 0x3ffb1d88  A12     : 0x3ffdb9ac  A13     : 0x3ffda660  
[19:27:18]A14     : 0x3ffdb92c  A15     : 0x3ffda2f0  SAR     : 0x0000000a  EXCCAUSE: 0x0000001c  
[19:27:18]EXCVADDR: 0x00000010  LBEG    : 0x400d55e0  LEND    : 0x400d55e8  LCOUNT  : 0x00000000  
[19:27:18]
[19:27:18]ELF file SHA256: 0000000000000000
[19:27:18]
[19:27:18]Backtrace: 0x400d6026:0x3ffb1cf0 0x400d60f5:0x3ffb1d20 0x400d6377:0x3ffb1d70 0x400d5989:0x3ffb1ea0 0x400d5b31:0x3ffb1ef0 0x4017aebd:0x3ffb1f20 0x4017af51:0x3ffb1f40 0x400d9a8e:0x3ffb1f60 0x400dbabe:0x3ffb1f90 0x400dfcc9:0x3ffb1fb0 0x40089cb6:0x3ffb1fd0
WARNING Found stack trace! Trying to decode it
WARNING Decoded 0x400d6026: std::vector<unsigned char, std::allocator<unsigned char> >::size() const at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/stl_vector.h:655
 (inlined by) std::vector<unsigned char, std::allocator<unsigned char> >::operator=(std::vector<unsigned char, std::allocator<unsigned char> > const&) at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:191
WARNING Decoded 0x400d60f5: esphome::sml::ObisInfo::ObisInfo(std::vector<unsigned char, std::allocator<unsigned char> >, esphome::sml::SmlNode) at /config/stromzaehler/src/esphome/components/sml/sml_parser.cpp:150
WARNING Decoded 0x400d6377: esphome::sml::SmlFile::get_obis_info() at /config/stromzaehler/src/esphome/components/sml/sml_parser.cpp:69 (discriminator 3)
WARNING Decoded 0x400d5989: esphome::sml::Sml::process_sml_file_(std::vector<unsigned char, std::allocator<unsigned char> > const&) at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:101
WARNING Decoded 0x400d5b31: esphome::sml::Sml::loop() at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:101
WARNING Decoded 0x4017aebd: esphome::Component::call_loop() at /config/stromzaehler/src/esphome/core/component.cpp:121
WARNING Decoded 0x4017af51: esphome::Component::call() at /config/stromzaehler/src/esphome/core/component.cpp:121
WARNING Decoded 0x400d9a8e: esphome::Application::loop() at /config/stromzaehler/src/esphome/core/application.cpp:74
WARNING Decoded 0x400dbabe: loop() at /config/stromzaehler/src/main.cpp:132
WARNING Decoded 0x400dfcc9: loopTask(void*) at /root/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:23
WARNING Decoded 0x40089cb6: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)
[19:27:18]
[19:27:18]Rebooting...
[19:27:18]ets Jun  8 2016 00:22:57
[19:27:18]
[19:27:18]rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[19:27:18]configsip: 0, SPIWP:0xee
[19:27:18]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[19:27:18]mode:DIO, clock div:2
[19:27:18]load:0x3fff0018,len:4
[19:27:18]load:0x3fff001c,len:1044
[19:27:18]load:0x40078000,len:10124
[19:27:18]load:0x40080400,len:5828
[19:27:18]entry 0x400806a8
[19:27:18][I][logger:174]: Log initialized
[19:27:18][C][ota:379]: There have been 2 suspected unsuccessful boot attempts.
[19:27:18][I][app:029]: Running through setup()...
[19:27:18][C][uart_esp32:072]: Setting up UART...
[19:27:18][C][wifi:037]: Setting up WiFi...
[19:27:18][D][wifi:382]: Starting scan...
[19:27:19]Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
[19:27:19]Core 1 register dump:
[19:27:19]PC      : 0x400d6026  PS      : 0x00060630  A0      : 0x800d60f8  A1      : 0x3ffb1bf0  
WARNING Decoded 0x400d6026: std::vector<unsigned char, std::allocator<unsigned char> >::size() const at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/stl_vector.h:655
 (inlined by) std::vector<unsigned char, std::allocator<unsigned char> >::operator=(std::vector<unsigned char, std::allocator<unsigned char> > const&) at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:191
[19:27:19]A2      : 0x3ffb1d54  A3      : 0x0000000c  A4      : 0x00000000  A5      : 0x3ffb1d3c  
[19:27:19]A6      : 0x00000000  A7      : 0x00000000  A8      : 0x800d60b4  A9      : 0x3ffb1bd0  
[19:27:19]A10     : 0x3ffb1bf4  A11     : 0x3ffb1c88  A12     : 0x3ffdab08  A13     : 0x3ffd97bc  
[19:27:19]A14     : 0x3ffdaa88  A15     : 0x3ffd944c  SAR     : 0x0000000a  EXCCAUSE: 0x0000001c  
[19:27:19]EXCVADDR: 0x00000010  LBEG    : 0x400d55e0  LEND    : 0x400d55e8  LCOUNT  : 0x00000000  
[19:27:19]
[19:27:19]ELF file SHA256: 0000000000000000
[19:27:19]
[19:27:19]Backtrace: 0x400d6026:0x3ffb1bf0 0x400d60f5:0x3ffb1c20 0x400d6377:0x3ffb1c70 0x400d5989:0x3ffb1da0 0x400d5b31:0x3ffb1df0 0x4017aebd:0x3ffb1e20 0x4017af51:0x3ffb1e40 0x400da975:0x3ffb1e60 0x400dbea9:0x3ffb1eb0 0x400dfcba:0x3ffb1fb0 0x40089cb6:0x3ffb1fd0
WARNING Found stack trace! Trying to decode it
WARNING Decoded 0x400d6026: std::vector<unsigned char, std::allocator<unsigned char> >::size() const at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/stl_vector.h:655
 (inlined by) std::vector<unsigned char, std::allocator<unsigned char> >::operator=(std::vector<unsigned char, std::allocator<unsigned char> > const&) at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:191
WARNING Decoded 0x400d60f5: esphome::sml::ObisInfo::ObisInfo(std::vector<unsigned char, std::allocator<unsigned char> >, esphome::sml::SmlNode) at /config/stromzaehler/src/esphome/components/sml/sml_parser.cpp:150
WARNING Decoded 0x400d6377: esphome::sml::SmlFile::get_obis_info() at /config/stromzaehler/src/esphome/components/sml/sml_parser.cpp:69 (discriminator 3)
WARNING Decoded 0x400d5989: esphome::sml::Sml::process_sml_file_(std::vector<unsigned char, std::allocator<unsigned char> > const&) at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:101
WARNING Decoded 0x400d5b31: esphome::sml::Sml::loop() at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:101
WARNING Decoded 0x4017aebd: esphome::Component::call_loop() at /config/stromzaehler/src/esphome/core/component.cpp:121
WARNING Decoded 0x4017af51: esphome::Component::call() at /config/stromzaehler/src/esphome/core/component.cpp:121
WARNING Decoded 0x400da975: esphome::Application::setup() at /config/stromzaehler/src/esphome/core/application.cpp:50 (discriminator 2)
WARNING Decoded 0x400dbea9: setup() at /config/stromzaehler/src/main.cpp:127
WARNING Decoded 0x400dfcba: loopTask(void*) at /root/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:18
WARNING Decoded 0x40089cb6: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)
[19:27:19]
[19:27:19]Rebooting...
[19:27:19]ets Jun  8 2016 00:22:57
[19:27:19]
[19:27:19]rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[19:27:19]configsip: 0, SPIWP:0xee
[19:27:19]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[19:27:19]mode:DIO, clock div:2
[19:27:19]load:0x3fff0018,len:4
[19:27:19]load:0x3fff001c,len:1044
[19:27:19]load:0x40078000,len:10124
[19:27:19]load:0x40080400,len:5828
[19:27:19]entry 0x400806a8
[19:27:19][I][logger:174]: Log initialized
[19:27:19][C][ota:379]: There have been 3 suspected unsuccessful boot attempts.

I get a LoadProhibited panic with the address 0x00000010. It seems that a wrong memory is is getting accessed. The ESP remains then in a boot loop.

In the stack trace it can be seen, that the crash happens in the std::allocator function. So I guess the ESP runs out of memory. @alengwenus what do you think?

alengwenus commented 2 years ago

I don't have any idea from the stack trace. Looks strange. It seems to happen after the parsing is already done (which should be the most memory intensive operation). However, do you have access to the raw data which is sent by your energy meter @BMOD89? I optimized the code to be more memory efficient (also around the mentioned position). But I doubt it will solve your problem. You might want to test it here. Too bad I can't reproduce it.

BMOD89 commented 2 years ago

Unfortunately it doesn't help: 

INFO Successfully uploaded program.
INFO Starting log output from /dev/ttyUSB1 with baud rate 115200
[04:39:36][I][logger:174]: Log initialized
[04:39:36][C][ota:379]: There have been 0 suspected unsuccessful boot attempts.
[04:39:36][I][app:029]: Running through setup()...
[04:39:36][C][uart_esp32:072]: Setting up UART...
[04:39:36][C][wifi:037]: Setting up WiFi...
[04:39:36][D][wifi:382]: Starting scan...
[04:39:37]Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
[04:39:37]Core 1 register dump:
[04:39:37]PC      : 0x400d5d72  PS      : 0x00060630  A0      : 0x800d5f00  A1      : 0x3ffb1bf0  
INFO Need to fetch platformio IDE-data, please stand by
INFO Running:  platformio run -d stromzaehler -t idedata
WARNING Decoded 0x400d5d72: std::vector<unsigned char, std::allocator<unsigned char> >::size() const at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/stl_vector.h:655
 (inlined by) std::vector<unsigned char, std::allocator<unsigned char> >::operator=(std::vector<unsigned char, std::allocator<unsigned char> > const&) at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:191
[04:39:38]A2      : 0x3ffb1d54  A3      : 0x0000000c  A4      : 0x00000000  A5      : 0x3ffb1d3c  
[04:39:38]A6      : 0x00000000  A7      : 0x00000000  A8      : 0x800d5df1  A9      : 0x3ffb1be0  
[04:39:38]A10     : 0x3ffb1d48  A11     : 0x3ffb1c88  A12     : 0x3ffd9454  A13     : 0x3ffd8108  
[04:39:38]A14     : 0x3ffd93d4  A15     : 0x3ffd7d98  SAR     : 0x0000000a  EXCCAUSE: 0x0000001c  
[04:39:38]EXCVADDR: 0x00000010  LBEG    : 0x400d5418  LEND    : 0x400d5420  LCOUNT  : 0x00000000  
[04:39:38]
[04:39:38]ELF file SHA256: 0000000000000000
[04:39:38]
[04:39:38]Backtrace: 0x400d5d72:0x3ffb1bf0 0x400d5efd:0x3ffb1c20 0x400d6097:0x3ffb1c70 0x400d57d1:0x3ffb1da0 0x400d5979:0x3ffb1df0 0x401573d9:0x3ffb1e20 0x4015746d:0x3ffb1e40 0x400da65d:0x3ffb1e60 0x400dbb69:0x3ffb1eb0 0x400df90a:0x3ffb1fb0 0x40089cb6:0x3ffb1fd0
WARNING Found stack trace! Trying to decode it
WARNING Decoded 0x400d5d72: std::vector<unsigned char, std::allocator<unsigned char> >::size() const at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/stl_vector.h:655
 (inlined by) std::vector<unsigned char, std::allocator<unsigned char> >::operator=(std::vector<unsigned char, std::allocator<unsigned char> > const&) at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:191
WARNING Decoded 0x400d5efd: esphome::sml::ObisInfo::ObisInfo(std::vector<unsigned char, std::allocator<unsigned char> >, esphome::sml::SmlNode) at /config/stromzaehler/src/esphome/components/sml/sml_parser.cpp:147
WARNING Decoded 0x400d6097: esphome::sml::SmlFile::get_obis_info() at /config/stromzaehler/src/esphome/components/sml/sml_parser.cpp:64 (discriminator 3)
WARNING Decoded 0x400d57d1: esphome::sml::Sml::process_sml_file_(std::vector<unsigned char, std::allocator<unsigned char> > const&) at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:101
WARNING Decoded 0x400d5979: esphome::sml::Sml::loop() at /root/.platformio/packages/toolchain-xtensa32/xtensa-esp32-elf/include/c++/5.2.0/bits/vector.tcc:101
WARNING Decoded 0x401573d9: esphome::Component::call_loop() at /config/stromzaehler/src/esphome/core/component.cpp:121
WARNING Decoded 0x4015746d: esphome::Component::call() at /config/stromzaehler/src/esphome/core/component.cpp:121
WARNING Decoded 0x400da65d: esphome::Application::setup() at /config/stromzaehler/src/esphome/core/application.cpp:50 (discriminator 2)
WARNING Decoded 0x400dbb69: setup() at /config/stromzaehler/src/main.cpp:127
WARNING Decoded 0x400df90a: loopTask(void*) at /root/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:18
WARNING Decoded 0x40089cb6: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)
[04:39:38]
[04:39:38]Rebooting...

I have sent you some raw data of my meter by mail.

Thanks!

BMOD89 commented 2 years ago

It works now like a charm. Thanks a lot!

yan14 commented 2 years ago

I tried to start over using the TeleInfo component as a template. Finally I gave up and started from scratch. 🙄 Well, now I come around with a more general approach where we can get the values by specifying the OBIS codes (https://github.com/alengwenus/esphome_components). A typical configuration for this solution looks like:

uart:
  id: uart_bus
  tx_pin: GPIO1
  rx_pin: GPIO3
  baud_rate: 9600

sml:
  uart_id: uart_bus
  logging: false

sensor:
 - platform: sml
    name: "Total energy"
    obis: "1-0:1.8.0"
    unit_of_measurement: kWh
    accuracy_decimals: 1
    filters:
      - multiply: 0.0001

text_sensor:
  - platform: sml
    name: "Manufacturer"
    obis: "129-129:199.130.3"
    format: text

For this I wrote a minimal SML parser which is integrated into the component. To get the available OBIS codes there is a HASS event based logging mechanism available. I also spent an hour to write some documentation. Would be great if someone could test and/or give some feedback if this approach is heading into the right direction. The whole stuff is getting more and more complicated, so don't be too hard on me if it's not perfectly mature. But it works quite well for my EMH smart meter.

Your solution looks really good. I want to test it in the next couple of days. What hardware did you use?

autox86 commented 2 years ago

Any ESP is fine to use as long as you can connect to the pins. Reader I recommend you this one: Hitchi Lesekopf

yan14 commented 2 years ago

Thanks, so this won‘t work? https://www.ebay.de/itm/Licht-Light-Helligkeits-3-4-Pin-Diode-Sensor-Modul-LM393-N-3-5V-ESP8266-Arduino-/284109007684?mkcid=16&mkevt=1&_trksid=p2349624.m46890.l49286&mkrid=707-127634-2357-0

autox86 commented 2 years ago

For simple "Light sense" reading, maybe

For SML., I assume no.

But not sure, maybe check out the volkszaehler.org wiki. There is a lot of info about hardware for such.

tunip commented 2 years ago

Many thanks for this implementation. Works fine so far with my meter. Would like to ask if you could add update_interval to the sensor for Active power. My meter is sending every 1-2 seconds a new value. Every 15-30 seconds would be enough for my use case.

syssi commented 2 years ago

@tunip You have to apply an throttle filter here. The update interval is available for polling components only (by design).

yan14 commented 2 years ago

With the lightsensor, I got no luck yesterday. So I bought the one you, @cheisig, recomended. It arrived today and it works perfectly. Now I got one question and one "change" to the example code

First the question. My smart meter can show the 'Current active power' but you need to enter the code. Is it possible to add the paramter and sent the code to the smart meter?

Second the example code. If the device_class and state_class it added to the sensor, Homeassistant Engery can use the sensors, too. Here is my exampe for the sensor:

sensor:
  - platform: sml
    name: "${friendly_name} - Total"
    sml_id: mysml 
    obis: "1-0:1.8.0"
    unit_of_measurement: kWh
    device_class: energy
    state_class: total_increasing
    accuracy_decimals: 1
    filters:
      - multiply: 0.0001

  - platform: sml
    name: "${friendly_name} - Current active power"
    sml_id: mysml
    obis: "1-0:16.7.0"
    unit_of_measurement: W
    accuracy_decimals: 1
    state_class: measurement
    device_class: power
    filters:
      - multiply: 0.1

text_sensor:
  - platform: version
    hide_timestamp: true
    name: "${friendly_name} - ESPHome Version"

  - platform: wifi_info
    ip_address:
      name: "${friendly_name} - IP Address"
      icon: mdi:wifi
    ssid:
      name: "${friendly_name} - Connected SSID"
      icon: mdi:wifi-strength-2
jarny0815 commented 2 years ago

Yesterday I am trying the current version (4.okt 2021) from alengwenus SML component for the first time. In the logoutput I see the obis records from my landis+gyr E320 counter. Nearly every second I get those records: [10:34:11][D][sml:085]: (0a014c475a00035b05a4) 1-0:96.50.1 [0x4c475a] [10:34:11][D][sml:085]: (0a014c475a00035b05a4) 1-0:96.1.0

1-0:1.8.0 [0x0000000001c2778c] [10:34:11][D][sml:085]: (0a014c475a00035b05a4) 1-0:16.7.0 [0x0000000000000130] The values seems correct. I convert the hex value from the log to decimal values. Next step is to receive the records from obis 1.8.0 and 16.7.1 in my home assistant installation. I tried this configuration with no success. The sensors 'Total energy' and 'Active power' do not show any value in homeassistant:

external_components:

esphome: name: devboard2 platform: ESP32 board: nodemcu-32s

includes:

- uart_read_line_sensor.h

wifi: ssid: "xxx" password: "xxxxxxxxxxx"

Enable fallback hotspot (captive portal) in case wifi connection fails

ap: ssid: "Devboard2 Fallback Hotspot" password: "xxxxxxxxxxxx"

captive_portal:

Enable logging

logger: level: DEBUG baud_rate: 0

Enable Home Assistant API

api:

ota:

uart: id: uart_bus tx_pin: GPIO17 rx_pin: GPIO16

baud_rate: 9600 rx_buffer_size: 2048 data_bits: 8 stop_bits: 1 parity: NONE

sml: id: mysml uart_id: uart_bus

sensor:

Am Sa., 2. Okt. 2021 um 18:56 Uhr schrieb yan14 @.***>:

With the lightsensor, I got no luck yesterday. So I bought the one you, @cheisig https://github.com/cheisig, recomended. It arrived today and it works perfectly. Now I got one question and one "change" to the example code

First the question. My smart meter can show the 'Current active power' but you need to enter the code. Is it possible to add the paramter and sent the code to the smart meter?

Second the example code. If the device_class and state_class it added to the sensor, Homeassistant Engery can use the sensors, too. Here is my exampe for the sensor:

`sensor:

-

platform: sml name: "${friendly_name} - Total" sml_id: mysml obis: "1-0:1.8.0" unit_of_measurement: kWh device_class: energy state_class: total_increasing accuracy_decimals: 1 filters:

  • multiply: 0.0001

  • platform: sml name: "${friendly_name} - Current active power" sml_id: mysml obis: "1-0:16.7.0" unit_of_measurement: W accuracy_decimals: 1 state_class: measurement device_class: power filters:

  • multiply: 0.1

text_sensor:

-

platform: version hide_timestamp: true name: "${friendly_name} - ESPHome Version"

platform: wifi_info ip_address: name: "${friendly_name} - IP Address" icon: mdi:wifi ssid: name: "${friendly_name} - Connected SSID" icon: mdi:wifi-strength-2 `

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/esphome/feature-requests/issues/1041#issuecomment-932783761, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVGQCYGVHZP34YDWJPPK4ZDUE42TVANCNFSM4U5RZNQA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

alengwenus commented 2 years ago

@jarny0815 You should either remove the dummy server_id in your configuration or replace it with the server_id of your smart meter, which you can get from your logs (0a014c475a00035b05a4).