Closed Be-Virtual closed 2 years ago
probot-esphome[bot]
commented
2 years ago Hey there @glmnet, @zuidwijk, mind taking a look at this issue as it has been labeled with an integration (dsmr) you are listed as a code owner for? Thanks!
(message by CodeOwnersMention)
defl
commented
2 years ago I had other issues when upgrading to 2021.9.0 where the DSMR started to report corrupted data. All of my 2021.9.0 devices started to have issues with their sensors and connectivity so I did a full rollback to 2021.8 and all is fine again. This might not be related to DSMR itself but to ESPHome
DrCoolzic
commented
2 years ago I have a similar problem problem with teleinfo component: lots of information missing! see https://github.com/esphome/issues/issues/2435
OttoWinter
commented
2 years ago This might be fixed with esphome/esphome#2342 and I've heard reports that it fixes similar issues.
Please try again with 2021.9.1 (which includes that fix), thanks!
zuidwijk
commented
2 years ago I've noticed with v2021.9.0 that when accessing the webpage, the ping to the device (which I had open) drops... closing the webpage and the ping resumes after a while...
zuidwijk
commented
2 years ago Look better with 2021.9.1 indeed... Yet I also have the impression that it can be caused by a brownout. When the esp sends too much data via wifi, it reaches the 250mA limit from the meter.
mmakaay
commented
2 years ago I have a DSMR reader that runs on the power as delivered by my DSMR v4.2 meter, which it considerably lower than the 250mA for a v5 meter. I have no brownout issues with this one. I do log more dan normally, because I'm working on a UART debugger, which logs all received bytes. Therefore, my first impression is that brownouts shouldn't be the issue here.
I will pump up the logging to see if that results in network issues.
I have VERY_VERBOSE logging active now, which logs every serial byte read as binary + hex. So that's a LOT of logging for this device :-)
I do see an error [Errno 104] Connection reset by peer now and then from within the API logger client.
I'll leave this running for a while to see if the uptime sensor drops together with the network drops (which could indicate a brownout reboot).
The logger has stopped working. When I try to reconnect, I get:
INFO Reading configuration smartmeter.yaml...
INFO Starting log output from 192.168.100.176 using esphome API
WARNING Can't connect to ESPHome API for 192.168.100.176: Error connecting to ('192.168.100.176', 6053): [Errno 111] Connect call failed ('192.168.100.176', 6053)
INFO Trying to reconnect to 192.168.100.176 in the backgroundHowever, I can ping the device successfully.
And I can also flash the device, so I put back a config with my previous loglevel VERBOSE.
After flashing, I can reconnect to the API logger.
mmakaay
commented
2 years ago No wait... ignore all of the above. I'm not building using my rebased dev branch, but using a separate branch that I'm building in, whoops shame on me. But at least this gives me a really nice base line to check against :-)
mmakaay
commented
2 years ago My colleague reports that he had a device with some sensors that went off line a lot with 2021.9. This is a device that is powered by a power supply unit, so that can't be a brownout issue. After upgrading to 2021.9.1, the device has become stable (for about 20 minutes, which he hadn't seen before).
mmakaay
commented
2 years ago I'm now running my smart meter reader with VERY_VERBOSE active, and I've seen some interesting behavior:
INFO Successfully compiled program.
INFO Connecting to 192.168.100.176
INFO Uploading /config/smartmeter/.pioenvs/smartmeter/firmware.bin (392176 bytes)
Uploading: [====================================================== ] 90%
ERROR Error sending data: timed outAfter some retries, it worked out, and I've got a working device again.
I'll leave it running now without the excessive amount of logging that is triggered by VERY_VERBOSE. Note BTW that the UART debugger will help a lot here, because VERY_VERBOSE is only used to see the individual bytes that were read, but the UART debugger will do it in a more friendly way (not 1 log line / 1 byte, unless one configures it willingly as such).
I have the device running now with the stock firmware in VERBOSE log mode. While in 2021.8.* I have not seen issues with this, now I pretty soon got the message:
ConnectionResetError: [Errno 104] Connection reset by peer
INFO Disconnected from ESPHome API for 192.168.100.176
WARNING Disconnected from API
WARNING Can't connect to ESPHome API for 192.168.100.176: Error connecting to ('192.168.100.176', 6053): [Errno 111] Connect call failed ('192.168.100.176', 6053)
INFO Trying to reconnect to 192.168.100.176 in the backgroundAfter this message, the logger stays silent. No reconnect is done. Starting a fresh logger doesn't work either. After this, flashing the device does start, but again:
INFO Successfully compiled program.
INFO Connecting to 192.168.100.176
INFO Uploading /config/smartmeter/.pioenvs/smartmeter/firmware.bin (392176 bytes)
Uploading: [=========== ] 18%
ERROR Error sending data: [Errno 104] Connection reset by peerOn a second try, it did work.
I will now switch to WARN for the logger. That surpresses most of the standard logging. I'll leave the device running like that to see how it behaves.
mmakaay
commented
2 years ago With log level WARN, the device has been stable for 5 hours now.
jsuanet
commented
2 years ago My colleague reports that he had a device with some sensors that went off line a lot with 2021.9. This is a device that is powered by a power supply unit, so that can't be a brownout issue. After upgrading to 2021.9.1, the device has become stable (for about 20 minutes, which he hadn't seen before).
And this device is still stable after 16 hours 😀 Also my DSMR-device (dsmr-v5), powered by the P1 port, did not have any disconnect after the recompile with 2021.9.1.
mmakaay
commented
2 years ago Went back to VERY_VERBOSE (because, who wants a stable device?) and things went south again. I could ping and flash the device, but the API connections were refused.
What I tried after this, was to flash it back to WARN. Flashing succeeded, multiple times, but the strange thing is that the device stayed unavailable. I would suspect when a flash works, that the device restarts afterwards and will connect perfecly fine with the downtuned logging. After leavingi the device simmer for a while, it did come back online in its stable form. Weird, so I will do some more testing to see what the pattern is.
Now I'm flashing again with VERY_VERBOSE, but also with MQTT enabled, to make it possible to view some logs via MQTT. I'm curious to see if MQTT still works when the API is refusing connections.
Ah, here's something that results in the meter not publishing data:
[11:03:51][V][dsmr:043]: Footer found
[11:03:51][VV][uart_esp8266:154]: Read 0b00110101 (0x35)
[11:03:51][VV][uart_esp8266:154]: Read 0b01000010 (0x42)
[11:03:51][VV][uart_esp8266:154]: Read 0b01000110 (0x46)
[11:03:51][VV][uart_esp8266:154]: Read 0b00110010 (0x32)
[11:03:51][VV][uart_esp8266:154]: Read 0b00001101 (0x0D)
[11:03:51][VV][uart_esp8266:154]: Read 0b00001010 (0x0A)
[11:03:51][V][dsmr:131]: Trying to parse
[11:03:51][E][dsmr:138]: !5BF2
^
Checksum mismatch
[11:03:51][V][component:207]: Component dsmr took a long time for an operation (1.16 s).
[11:03:51][V][component:208]: Components should block for at most 20-30ms.
[11:03:51][VV][api.service:327]: on_ping_request: PingRequest {}
[11:03:51][VV][api.service:043]: send_ping_response: PingResponse {}I am getting checksum mismatches on the telegrams. Possibly missing bytes in the serial data that were read from the smart meter? With that reading time of 1.16s, I can see where things are clogging.
mmakaay
commented
2 years ago Okay, getting somewhere. One problem that we might be looking at for the DSMR component, is that its loop() is very greedy. It will not stop reading until the last available byte on the serial bus is read. Combined with VERY_VERBOSE logging, this causes the loop to block for a long time.
This might be even be more prevalent on a DSMR 5 reader, since those tend to provide data over serial a lot more frequently to DSMR 4 (I get updates every 15 seconds, while DSMR 5 is more like 1 to a few seconds I think). If the logging takes more time than the time gap between two data cycles, then this might result in an endlessly blocking loop().
I did a quick hack on my device (against the 2021.9.1 release version of ESPHome) to make that loop less greedy by letting it read a max number of bytes per loop() call, and that seems to help a lot. I now have a device running in VERY_VERBOSE mode with the hack, and it has been reporting values for the last 20 minutes without hickups. That's a new record for VERY_VERBOSE 🥇
I'm currently having issues with getting a working device using the dev branch, but as soon as that is figured out, I will provide a PR for fixing this issue.
Another thing that might be useful, is to allow the configuration of a pin that is used for steering the data request line of the P1 port. That request line must be pulled high to request telegrams from the meter. This can effectively be used for preventing overlapping telegrams, by dropping the request line while a telegram is being processed and pulling it up afterwards. Such request line would also allow for implementing a customized update frequency. Of course, the hardware must support the use of such request pin. Most circuits simply connect +5V to the data request line to have the meter send datagrams as long as the device is connected to the port.
mmakaay
commented
2 years ago The latest dev had some issues with ESP8266 (some big changes were merged and there was some collateral damage), so I put some time into ironing out bugs. I can now flash and run my devices successfully on dev again, so back to fixing some DSMR issues now.
defl
commented
2 years ago Just flashed my Slimmelezer to 2021.09.01:
[14:54:20][C][wifi_info:010]: WifiInfo SSID 'Wi-Fi SSID' [14:54:20][C][wifi_info:011]: WifiInfo BSSID 'Wi-Fi BSSID'
^ Checksum mismatch
^ Checksum mismatch
^ Checksum mismatch
^ Checksum mismatch
^ Checksum mismatch
Rolling back...
mmakaay
commented
2 years ago What log level are you at? I could push my device into these checksum mismatches while using VERY_VERBOSE. With that setting, I think that some bytes from the telegram body simply went missing for me, because the device was too busy with all the logging involved.
If it's not the log level (you could try and see if WARN as level fixes anything for you), then I'd still be interested to see if things will be fixed by the PR that I'm currently preparing.
defl
commented
2 years ago Just the default of "logger:" without any further config. Will try with WARN
defl
commented
2 years ago With WARN:
INFO Successfully compiled program.
INFO Connecting to 192.168.30.106
INFO Uploading /data/slimmelezer-k1/.pioenvs/slimmelezer-k1/firmware.bin (388576 bytes)
Uploading: [============================================================] 100% Done...
INFO Waiting for result...
INFO OTA successful
INFO Successfully uploaded program.
INFO Starting log output from 192.168.30.106 using esphome API
INFO Successfully connected to 192.168.30.106
[15:26:49][E][dsmr:138]: !C732
^
Checksum mismatch
[15:26:50][E][dsmr:138]: !5673
^
Checksum mismatch
[15:26:51][E][dsmr:138]: !ADE6
^
Checksum mismatch
[15:26:54][E][dsmr:138]: !97C1
^
Checksum mismatch
[15:26:55][E][dsmr:138]: !F3ADStill broken, so something else is interfering here. Are you able to run ESPHome from a dev branch of mine? It's the latest dev version, plus a few dsmr changes that I did.
And could you share your (password-redacted) YAML config, so I can try reproducing this on my device?
defl
commented
2 years ago @mmakaay yeah no problem, point me to it.
Config:
esphome:
name: slimmelezer-k1
platform: ESP8266
esp8266_restore_from_flash: true
board: d1_mini
name_add_mac_suffix: false
project:
name: zuidwijk.slimmelezer
version: "1.0"
wifi:
ssid: !secret wifi_iot_h_ssid
password: !secret wifi_iot_h_password
fast_connect: true # Required for hidden networks
use_address: 192.168.30.106
logger:
level: WARN
api:
ota:
uart:
baud_rate: 115200
rx_pin: D7
dsmr:
id: dsmr_instance
sensor:
- platform: dsmr
energy_delivered_tariff1:
name: "Energy Consumed Tariff 1"
state_class: total_increasing
energy_delivered_tariff2:
name: "Energy Consumed Tariff 2"
state_class: total_increasing
energy_returned_tariff1:
name: "Energy Produced Tariff 1"
state_class: total_increasing
energy_returned_tariff2:
name: "Energy Produced Tariff 2"
state_class: total_increasing
power_delivered:
name: "Power Consumed"
accuracy_decimals: 0
filters:
- multiply: 1000
power_returned:
name: "Power Produced"
accuracy_decimals: 0
filters:
- multiply: 1000
electricity_failures:
name: "Electricity Failures"
icon: mdi:alert
electricity_long_failures:
name: "Long Electricity Failures"
icon: mdi:alert
voltage_l1:
name: "Voltage Phase 1"
voltage_l2:
name: "Voltage Phase 2"
voltage_l3:
name: "Voltage Phase 3"
current_l1:
name: "Current Phase 1"
current_l2:
name: "Current Phase 2"
current_l3:
name: "Current Phase 3"
power_delivered_l1:
name: "Power Consumed Phase 1"
accuracy_decimals: 0
filters:
- multiply: 1000
power_delivered_l2:
name: "Power Consumed Phase 2"
accuracy_decimals: 0
filters:
- multiply: 1000
power_delivered_l3:
name: "Power Consumed Phase 3"
accuracy_decimals: 0
filters:
- multiply: 1000
power_returned_l1:
name: "Power Produced Phase 1"
accuracy_decimals: 0
filters:
- multiply: 1000
power_returned_l2:
name: "Power Produced Phase 2"
accuracy_decimals: 0
filters:
- multiply: 1000
power_returned_l3:
name: "Power Produced Phase 3"
accuracy_decimals: 0
filters:
- multiply: 1000
gas_delivered:
name: "Gas Consumed"
state_class: total_increasing
- platform: uptime
name: "Uptime"
- platform: wifi_signal
name: "slimmelezer_k1 WiFi Signal"
update_interval: 240s
text_sensor:
- platform: dsmr
identification:
name: "DSMR Identification"
p1_version:
name: "DSMR Version"
p1_version_be:
name: "DSMR Version Belgium"
- platform: wifi_info
ip_address:
name: "IP Address"
ssid:
name: "Wi-Fi SSID"
bssid:
name: "Wi-Fi BSSID"
- platform: version
name: "ESPHome Version"
hide_timestamp: trueYou might want to try disabling serial logging to enable hardware UART:
logger:
baud_rate: 0
level: DEBUGFurthermore, you have a lot of sensors defined. Shouldn't be a big issue and certainly not a soluution, but for debugging purposes, could you trim down those to a few and see if that changes the behavior?
I just committed some changes to my fork. It'd be great if you could test drive it:
https://github.com/mmakaay/esphome/tree/dsmr-clogging-fix
I've got my own reader device running rock solid using this, even with VERY_VERBOSE logging active.
defl
commented
2 years ago This is a trimmed down version of the recommended config for the Slimmelezer, I was assuming the UART there was for the dsmr_instance bit but I didn't check TBH.
Running the main 2021.9.1 with baud_rate: 0 solves the problem, no more errors.
mmakaay
commented
2 years ago Ah nice, then I can try reproducing the issue. Maybe I can add a warning / hint when checksum failures come up with the uart logger active.
mmakaay
commented
2 years ago Kind of disappointing. Using the same configuration (without baud_rate=0 for the logger) and 2021.9.1, I am unable to trigger the same issue :-/ This is on my own reader device BTW, based on an ESP-01. Do you have a DSMR v5 meter by any chance, @defl ? Those send more telegrams, so that might trigger the different behavior here.
mmakaay
commented
2 years ago Ah, the way to reproduce it for me, is to use VERY_VERBOSE logging along with UART logging active. Quite harsh, I will apologize to my device 😃
defl
commented
2 years ago @mmakaay Yes I've got a DSMR v5 and it sends a LOT of messages (to the point where i'd actually appreciate a throttle in the dsmr module so that I don't need to put a filter per sensor in as I'm planning to do once this settles)
[17:45:43][D][text_sensor:015]: 'DSMR Identification': Sending state 'ISK5\2M550T-1012'
[17:45:43][D][text_sensor:015]: 'DSMR Version': Sending state '50'The best way to throttle would require hardware support. I'm currently extending my design with an option to drive the request pin of the P1 meter, which would make it possible to fully silence the meter when no data has to be read (e.g. during processing of a datagram or when a lower update frequency is required). Another way would be for the code to simply ignore incoming bytes for a given amount of time. That can be done fully in software. I'll see if I can add a feature for both routes.
On my device, I can enable the uart logger as long as I keep the log level at DEBUG or VERBOSE. But my v4 meter sends less data I think, and only once per 15 seconds. So that is likely the difference here.
When pushing the logger to its limits, I get the same error messages as you, most likely indicating dropped bytes on the serial line. The best fix for that is what we already found out: make sure that hardware UART is used.
Oh, and one thing that you don't need in the config, is the multiply: 1000. There were issues with kW versus W notations in the dsmr component output, but with the 2021.9.1, you can drop the multiplications in there. The reported values match the presented units.
Be-Virtual
commented
2 years ago Hi, As belgian user we need this piece of code to avoid the checksum error: Could help with you to.
esphome: platformio_options: lib_deps: glmnet/Dsmr@0.5 lib_ignore: glmnet/Dsmr@0.3
Till the update is taken into the prod version.
mmakaay
commented
2 years ago The checksum fix in the Dsmr lib probably is not the issue here, since switching to hardware UART fixed the issue. That means that the checksum errors most likely pointed at actual issues in reading the data.
michaelarnauts
commented
2 years ago I'm using ESPHome 2021.9.1.
I saw a lot of checksum errors in the logs, but those are gone now when setting baud_rate to 0 for logging.
logger:
baud_rate: 0I have a Belgian DSMRv5 meter, and get updates every second. Note that I had to use glmnet/Dsmr@0.5 like so.
platformio_options:
lib_deps: glmnet/Dsmr@0.5
lib_ignore: glmnet/Dsmr@0.3Also mentioned above, but I also had to remove the multiple: 1000. The values are correct as is
mmakaay
commented
2 years ago The 0.5 version of the Dsmr library will be used by default in 2021.10.0. One thing that I want to look at, is incorporating the parser in the ESPHome code tree directly, to prevent having to do library juggling like described above.
rhvs
commented
2 years ago Even with the dev branch I keep getting errors like these in the log:
Checksum mismatch
I used the yaml below and I'm totally out of options what to test next to get it working...
substitutions:
device_name: slimmelezer
device_description: "DIY P1 module to read your smart meter"
esphome:
name: ${device_name}
comment: "${device_description}"
platform: ESP8266
esp8266_restore_from_flash: true
board: d1_mini
name_add_mac_suffix: false
project:
name: zuidwijk.slimmelezer
version: "1.0"
on_boot:
then:
- if:
condition:
lambda: return id(has_key);
then:
- lambda: |-
std::string key(id(stored_decryption_key), 32);
id(dsmr_instance).set_decryption_key(key);
else:
- logger.log:
level: info
format: "Not using decryption key. If you need to set a key use Home Assistant service 'ESPHome: ${device_name}_set_dsmr_key'"
wifi:
networks:
- ssid: !secret wifi_ssid
password: !secret wifi_password
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: ${device_name}
ap_timeout: 15s
captive_portal:
# Enable logging
logger:
# Enable Home Assistant API
api:
services:
service: set_dsmr_key
variables:
private_key: string
then:
- logger.log:
format: Setting private key %s. Set to empty string to disable
args: [private_key.c_str()]
- globals.set:
id: has_key
value: !lambda "return private_key.length() == 32;"
- lambda: |-
if (private_key.length() == 32)
private_key.copy(id(stored_decryption_key), 32);
id(dsmr_instance).set_decryption_key(private_key);
ota:
web_server:
port: 80
uart:
baud_rate: 115200
rx_pin: D7
globals:
- id: has_key
type: bool
restore_value: yes
initial_value: "false"
- id: stored_decryption_key
type: char[32]
restore_value: yes
dsmr:
id: dsmr_instance
# For Luxembourg users set here your decryption key.
# Enable this when using decryption for Luxembourg;
# key looks like '00112233445566778899AABBCCDDEEFF'
#decryption_key: !secret decryption_key
sensor:
- platform: dsmr
energy_delivered_lux:
name: "Energy Consumed Luxembourg"
state_class: total_increasing
energy_delivered_tariff1:
name: "Energy Consumed Tariff 1"
state_class: total_increasing
energy_delivered_tariff2:
name: "Energy Consumed Tariff 2"
state_class: total_increasing
energy_returned_lux:
name: "Energy Produced Luxembourg"
state_class: total_increasing
energy_returned_tariff1:
name: "Energy Produced Tariff 1"
state_class: total_increasing
energy_returned_tariff2:
name: "Energy Produced Tariff 2"
state_class: total_increasing
power_delivered:
name: "Power Consumed"
accuracy_decimals: 3
power_returned:
name: "Power Produced"
accuracy_decimals: 3
electricity_failures:
name: "Electricity Failures"
icon: mdi:alert
electricity_long_failures:
name: "Long Electricity Failures"
icon: mdi:alert
voltage_l1:
name: "Voltage Phase 1"
voltage_l2:
name: "Voltage Phase 2"
voltage_l3:
name: "Voltage Phase 3"
current_l1:
name: "Current Phase 1"
current_l2:
name: "Current Phase 2"
current_l3:
name: "Current Phase 3"
power_delivered_l1:
name: "Power Consumed Phase 1"
accuracy_decimals: 3
power_delivered_l2:
name: "Power Consumed Phase 2"
accuracy_decimals: 3
power_delivered_l3:
name: "Power Consumed Phase 3"
accuracy_decimals: 3
power_returned_l1:
name: "Power Produced Phase 1"
accuracy_decimals: 3
power_returned_l2:
name: "Power Produced Phase 2"
accuracy_decimals: 3
power_returned_l3:
name: "Power Produced Phase 3"
accuracy_decimals: 3
gas_delivered:
name: "Gas Consumed"
state_class: total_increasing
gas_delivered_be:
name: "Gas Consumed Belgium"
state_class: total_increasing
- platform: uptime
name: "SlimmeLezer Uptime"
- platform: wifi_signal
name: "SlimmeLezer Wi-Fi Signal"
update_interval: 60s
text_sensor:
- platform: dsmr
identification:
name: "DSMR Identification"
p1_version:
name: "DSMR Version"
p1_version_be:
name: "DSMR Version Belgium"
- platform: wifi_info
ip_address:
name: "SlimmeLezer IP Address"
ssid:
name: "SlimmeLezer Wi-Fi SSID"
bssid:
name: "SlimmeLezer Wi-Fi BSSID"
- platform: version
name: "ESPHome Version"
hide_timestamp: trueAnd the output from ESPHome/HA:
INFO Reading configuration /config/esphome/slimmelezer.yaml...
INFO Generating C++ source...
INFO Compiling app...
INFO Running: platformio run -d /config/esphome/slimmelezer
Processing slimmelezer (board: d1_mini; framework: arduino; platform: platformio/espressif8266 @ 2.6.2)
--------------------------------------------------------------------------------
HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash
PACKAGES:
- framework-arduinoespressif8266 3.20704.0 (2.7.4)
- tool-esptool 1.413.0 (4.13)
- tool-esptoolpy 1.20800.0 (2.8.0)
- toolchain-xtensa 2.40802.200502 (4.8.2)
Library Manager: Installing Update
Warning! Could not find the package with 'Update' requirements for your system 'linux_aarch64'
Dependency Graph
|-- <ESPAsyncTCP-esphome> 1.2.3
| |-- <ESP8266WiFi> 1.0
|-- <ESPAsyncWebServer-esphome> 1.3.0
| |-- <ESPAsyncTCP-esphome> 1.2.3
| | |-- <ESP8266WiFi> 1.0
| |-- <Hash> 1.0
| |-- <Crypto> 0.2.0
| |-- <ESP8266WiFi> 1.0
| |-- <ArduinoJson-esphomelib> 5.13.3
|-- <ESP8266WiFi> 1.0
|-- <ArduinoJson-esphomelib> 5.13.3
|-- <Dsmr> 0.5.0
|-- <Crypto> 0.2.0
|-- <ESP8266mDNS> 1.2
| |-- <ESP8266WiFi> 1.0
|-- <noise-c> 0.1.3
| |-- <libsodium> 1.10018.1
|-- <DNSServer> 1.1.1
| |-- <ESP8266WiFi> 1.0
Compiling /data/slimmelezer/.pioenvs/slimmelezer/src/main.cpp.o
Linking /data/slimmelezer/.pioenvs/slimmelezer/firmware.elf
Building /data/slimmelezer/.pioenvs/slimmelezer/firmware.bin
Retrieving maximum program size /data/slimmelezer/.pioenvs/slimmelezer/firmware.elf
Checking size /data/slimmelezer/.pioenvs/slimmelezer/firmware.elf
RAM: [==== ] 41.1% (used 33668 bytes from 81920 bytes)
Flash: [==== ] 44.0% (used 459848 bytes from 1044464 bytes)
Creating BIN file "/data/slimmelezer/.pioenvs/slimmelezer/firmware.bin" using "/data/cache/platformio/packages/framework-arduinoespressif8266/bootloaders/eboot/eboot.elf" and "/data/slimmelezer/.pioenvs/slimmelezer/firmware.elf"
========================= [SUCCESS] Took 21.81 seconds =========================
INFO Successfully compiled program.
INFO Running: esptool.py --before default_reset --after hard_reset --baud 460800 --port /dev/ttyUSB3 --chip esp8266 write_flash -z 0x0 /data/slimmelezer/.pioenvs/slimmelezer/firmware.bin
esptool.py v3.1
Serial port /dev/ttyUSB3
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: c4:5b:be:6c:b3:e7
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00071fff...
Compressed 464000 bytes to 317488...
Wrote 464000 bytes (317488 compressed) at 0x00000000 in 7.4 seconds (effective 499.8 kbit/s)...
Hash of data verified.
Leaving...
Hard resetting via RTS pin...
INFO Successfully uploaded program.
INFO Starting log output from /dev/ttyUSB3 with baud rate 115200Even with the dev branch I keep getting errors like these in the log:
Checksum mismatch� [E][dsmr:154]: !7B12
I used the yaml below and I'm totally out of options what to test next to get it working...
substitutions: device_name: slimmelezer device_description: "DIY P1 module to read your smart meter" esphome: name: ${device_name} comment: "${device_description}" platform: ESP8266 esp8266_restore_from_flash: true board: d1_mini name_add_mac_suffix: false project: name: zuidwijk.slimmelezer version: "1.0" on_boot: then: - if: condition: lambda: return id(has_key); then: - lambda: |- std::string key(id(stored_decryption_key), 32); id(dsmr_instance).set_decryption_key(key); else: - logger.log: level: info format: "Not using decryption key. If you need to set a key use Home Assistant service 'ESPHome: ${device_name}_set_dsmr_key'" wifi: networks: - ssid: !secret wifi_ssid password: !secret wifi_password # Enable fallback hotspot (captive portal) in case wifi connection fails ap: ssid: ${device_name} ap_timeout: 15s captive_portal: # Enable logging logger: # Enable Home Assistant API api: services: service: set_dsmr_key variables: private_key: string then: - logger.log: format: Setting private key %s. Set to empty string to disable args: [private_key.c_str()] - globals.set: id: has_key value: !lambda "return private_key.length() == 32;" - lambda: |- if (private_key.length() == 32) private_key.copy(id(stored_decryption_key), 32); id(dsmr_instance).set_decryption_key(private_key); ota: web_server: port: 80 uart: baud_rate: 115200 rx_pin: D7 globals: - id: has_key type: bool restore_value: yes initial_value: "false" - id: stored_decryption_key type: char[32] restore_value: yes dsmr: id: dsmr_instance # For Luxembourg users set here your decryption key. # Enable this when using decryption for Luxembourg; # key looks like '00112233445566778899AABBCCDDEEFF' #decryption_key: !secret decryption_key sensor: - platform: dsmr energy_delivered_lux: name: "Energy Consumed Luxembourg" state_class: total_increasing energy_delivered_tariff1: name: "Energy Consumed Tariff 1" state_class: total_increasing energy_delivered_tariff2: name: "Energy Consumed Tariff 2" state_class: total_increasing energy_returned_lux: name: "Energy Produced Luxembourg" state_class: total_increasing energy_returned_tariff1: name: "Energy Produced Tariff 1" state_class: total_increasing energy_returned_tariff2: name: "Energy Produced Tariff 2" state_class: total_increasing power_delivered: name: "Power Consumed" accuracy_decimals: 3 power_returned: name: "Power Produced" accuracy_decimals: 3 electricity_failures: name: "Electricity Failures" icon: mdi:alert electricity_long_failures: name: "Long Electricity Failures" icon: mdi:alert voltage_l1: name: "Voltage Phase 1" voltage_l2: name: "Voltage Phase 2" voltage_l3: name: "Voltage Phase 3" current_l1: name: "Current Phase 1" current_l2: name: "Current Phase 2" current_l3: name: "Current Phase 3" power_delivered_l1: name: "Power Consumed Phase 1" accuracy_decimals: 3 power_delivered_l2: name: "Power Consumed Phase 2" accuracy_decimals: 3 power_delivered_l3: name: "Power Consumed Phase 3" accuracy_decimals: 3 power_returned_l1: name: "Power Produced Phase 1" accuracy_decimals: 3 power_returned_l2: name: "Power Produced Phase 2" accuracy_decimals: 3 power_returned_l3: name: "Power Produced Phase 3" accuracy_decimals: 3 gas_delivered: name: "Gas Consumed" state_class: total_increasing gas_delivered_be: name: "Gas Consumed Belgium" state_class: total_increasing - platform: uptime name: "SlimmeLezer Uptime" - platform: wifi_signal name: "SlimmeLezer Wi-Fi Signal" update_interval: 60s text_sensor: - platform: dsmr identification: name: "DSMR Identification" p1_version: name: "DSMR Version" p1_version_be: name: "DSMR Version Belgium" - platform: wifi_info ip_address: name: "SlimmeLezer IP Address" ssid: name: "SlimmeLezer Wi-Fi SSID" bssid: name: "SlimmeLezer Wi-Fi BSSID" - platform: version name: "ESPHome Version" hide_timestamp: trueAnd the output from ESPHome/HA:
INFO Reading configuration /config/esphome/slimmelezer.yaml... INFO Generating C++ source... INFO Compiling app... INFO Running: platformio run -d /config/esphome/slimmelezer Processing slimmelezer (board: d1_mini; framework: arduino; platform: platformio/espressif8266 @ 2.6.2) -------------------------------------------------------------------------------- HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash PACKAGES: - framework-arduinoespressif8266 3.20704.0 (2.7.4) - tool-esptool 1.413.0 (4.13) - tool-esptoolpy 1.20800.0 (2.8.0) - toolchain-xtensa 2.40802.200502 (4.8.2) Library Manager: Installing Update Warning! Could not find the package with 'Update' requirements for your system 'linux_aarch64' Dependency Graph |-- <ESPAsyncTCP-esphome> 1.2.3 | |-- <ESP8266WiFi> 1.0 |-- <ESPAsyncWebServer-esphome> 1.3.0 | |-- <ESPAsyncTCP-esphome> 1.2.3 | | |-- <ESP8266WiFi> 1.0 | |-- <Hash> 1.0 | |-- <Crypto> 0.2.0 | |-- <ESP8266WiFi> 1.0 | |-- <ArduinoJson-esphomelib> 5.13.3 |-- <ESP8266WiFi> 1.0 |-- <ArduinoJson-esphomelib> 5.13.3 |-- <Dsmr> 0.5.0 |-- <Crypto> 0.2.0 |-- <ESP8266mDNS> 1.2 | |-- <ESP8266WiFi> 1.0 |-- <noise-c> 0.1.3 | |-- <libsodium> 1.10018.1 |-- <DNSServer> 1.1.1 | |-- <ESP8266WiFi> 1.0 Compiling /data/slimmelezer/.pioenvs/slimmelezer/src/main.cpp.o Linking /data/slimmelezer/.pioenvs/slimmelezer/firmware.elf Building /data/slimmelezer/.pioenvs/slimmelezer/firmware.bin Retrieving maximum program size /data/slimmelezer/.pioenvs/slimmelezer/firmware.elf Checking size /data/slimmelezer/.pioenvs/slimmelezer/firmware.elf RAM: [==== ] 41.1% (used 33668 bytes from 81920 bytes) Flash: [==== ] 44.0% (used 459848 bytes from 1044464 bytes) Creating BIN file "/data/slimmelezer/.pioenvs/slimmelezer/firmware.bin" using "/data/cache/platformio/packages/framework-arduinoespressif8266/bootloaders/eboot/eboot.elf" and "/data/slimmelezer/.pioenvs/slimmelezer/firmware.elf" ========================= [SUCCESS] Took 21.81 seconds ========================= INFO Successfully compiled program. INFO Running: esptool.py --before default_reset --after hard_reset --baud 460800 --port /dev/ttyUSB3 --chip esp8266 write_flash -z 0x0 /data/slimmelezer/.pioenvs/slimmelezer/firmware.bin esptool.py v3.1 Serial port /dev/ttyUSB3 Connecting.... Chip is ESP8266EX Features: WiFi Crystal is 26MHz MAC: c4:5b:be:6c:b3:e7 Uploading stub... Running stub... Stub running... Changing baud rate to 460800 Changed. Configuring flash size... Flash will be erased from 0x00000000 to 0x00071fff... Compressed 464000 bytes to 317488... Wrote 464000 bytes (317488 compressed) at 0x00000000 in 7.4 seconds (effective 499.8 kbit/s)... Hash of data verified. Leaving... Hard resetting via RTS pin... INFO Successfully uploaded program. INFO Starting log output from /dev/ttyUSB3 with baud rate 115200
That code is new since yesterday (I changed the accuracy_decimals from 0 to 3). I have this with latest normal ESPHome (2021.9.3) working flawlessly on my test meter.
Do you get between those checksum mismatch also good readings? Those messages aren't necessarily bad, as a meter can spit out corrupt telegrams and than the checksum doesn't match.
rhvs
commented
2 years ago That code is new since yesterday (I changed the accuracy_decimals from 0 to 3). I have this with latest normal ESPHome (2021.9.3) working flawlessly on my test meter.
Do you get between those checksum mismatch also good readings? Those messages aren't necessarily bad, as a meter can spit out corrupt telegrams and than the checksum doesn't match.
Unfortunately nothing related to energy, just uptime and wifi signal.
-edit- perhaps meter info is relevant;
ISKRA AM550-ED1.04 SMR 5.0
svenahrensnl
commented
2 years ago Even with the dev branch I keep getting errors like these in the log: Checksum mismatch� [E][dsmr:154]: !7B12 I used the yaml below and I'm totally out of options what to test next to get it working...
substitutions: device_name: slimmelezer device_description: "DIY P1 module to read your smart meter" esphome: name: ${device_name} comment: "${device_description}" platform: ESP8266 esp8266_restore_from_flash: true board: d1_mini name_add_mac_suffix: false project: name: zuidwijk.slimmelezer version: "1.0" on_boot: then: - if: condition: lambda: return id(has_key); then: - lambda: |- std::string key(id(stored_decryption_key), 32); id(dsmr_instance).set_decryption_key(key); else: - logger.log: level: info format: "Not using decryption key. If you need to set a key use Home Assistant service 'ESPHome: ${device_name}_set_dsmr_key'" wifi: networks: - ssid: !secret wifi_ssid password: !secret wifi_password # Enable fallback hotspot (captive portal) in case wifi connection fails ap: ssid: ${device_name} ap_timeout: 15s captive_portal: # Enable logging logger: # Enable Home Assistant API api: services: service: set_dsmr_key variables: private_key: string then: - logger.log: format: Setting private key %s. Set to empty string to disable args: [private_key.c_str()] - globals.set: id: has_key value: !lambda "return private_key.length() == 32;" - lambda: |- if (private_key.length() == 32) private_key.copy(id(stored_decryption_key), 32); id(dsmr_instance).set_decryption_key(private_key); ota: web_server: port: 80 uart: baud_rate: 115200 rx_pin: D7 globals: - id: has_key type: bool restore_value: yes initial_value: "false" - id: stored_decryption_key type: char[32] restore_value: yes dsmr: id: dsmr_instance # For Luxembourg users set here your decryption key. # Enable this when using decryption for Luxembourg; # key looks like '00112233445566778899AABBCCDDEEFF' #decryption_key: !secret decryption_key sensor: - platform: dsmr energy_delivered_lux: name: "Energy Consumed Luxembourg" state_class: total_increasing energy_delivered_tariff1: name: "Energy Consumed Tariff 1" state_class: total_increasing energy_delivered_tariff2: name: "Energy Consumed Tariff 2" state_class: total_increasing energy_returned_lux: name: "Energy Produced Luxembourg" state_class: total_increasing energy_returned_tariff1: name: "Energy Produced Tariff 1" state_class: total_increasing energy_returned_tariff2: name: "Energy Produced Tariff 2" state_class: total_increasing power_delivered: name: "Power Consumed" accuracy_decimals: 3 power_returned: name: "Power Produced" accuracy_decimals: 3 electricity_failures: name: "Electricity Failures" icon: mdi:alert electricity_long_failures: name: "Long Electricity Failures" icon: mdi:alert voltage_l1: name: "Voltage Phase 1" voltage_l2: name: "Voltage Phase 2" voltage_l3: name: "Voltage Phase 3" current_l1: name: "Current Phase 1" current_l2: name: "Current Phase 2" current_l3: name: "Current Phase 3" power_delivered_l1: name: "Power Consumed Phase 1" accuracy_decimals: 3 power_delivered_l2: name: "Power Consumed Phase 2" accuracy_decimals: 3 power_delivered_l3: name: "Power Consumed Phase 3" accuracy_decimals: 3 power_returned_l1: name: "Power Produced Phase 1" accuracy_decimals: 3 power_returned_l2: name: "Power Produced Phase 2" accuracy_decimals: 3 power_returned_l3: name: "Power Produced Phase 3" accuracy_decimals: 3 gas_delivered: name: "Gas Consumed" state_class: total_increasing gas_delivered_be: name: "Gas Consumed Belgium" state_class: total_increasing - platform: uptime name: "SlimmeLezer Uptime" - platform: wifi_signal name: "SlimmeLezer Wi-Fi Signal" update_interval: 60s text_sensor: - platform: dsmr identification: name: "DSMR Identification" p1_version: name: "DSMR Version" p1_version_be: name: "DSMR Version Belgium" - platform: wifi_info ip_address: name: "SlimmeLezer IP Address" ssid: name: "SlimmeLezer Wi-Fi SSID" bssid: name: "SlimmeLezer Wi-Fi BSSID" - platform: version name: "ESPHome Version" hide_timestamp: trueAnd the output from ESPHome/HA:
INFO Reading configuration /config/esphome/slimmelezer.yaml... INFO Generating C++ source... INFO Compiling app... INFO Running: platformio run -d /config/esphome/slimmelezer Processing slimmelezer (board: d1_mini; framework: arduino; platform: platformio/espressif8266 @ 2.6.2) -------------------------------------------------------------------------------- HARDWARE: ESP8266 80MHz, 80KB RAM, 4MB Flash PACKAGES: - framework-arduinoespressif8266 3.20704.0 (2.7.4) - tool-esptool 1.413.0 (4.13) - tool-esptoolpy 1.20800.0 (2.8.0) - toolchain-xtensa 2.40802.200502 (4.8.2) Library Manager: Installing Update Warning! Could not find the package with 'Update' requirements for your system 'linux_aarch64' Dependency Graph |-- <ESPAsyncTCP-esphome> 1.2.3 | |-- <ESP8266WiFi> 1.0 |-- <ESPAsyncWebServer-esphome> 1.3.0 | |-- <ESPAsyncTCP-esphome> 1.2.3 | | |-- <ESP8266WiFi> 1.0 | |-- <Hash> 1.0 | |-- <Crypto> 0.2.0 | |-- <ESP8266WiFi> 1.0 | |-- <ArduinoJson-esphomelib> 5.13.3 |-- <ESP8266WiFi> 1.0 |-- <ArduinoJson-esphomelib> 5.13.3 |-- <Dsmr> 0.5.0 |-- <Crypto> 0.2.0 |-- <ESP8266mDNS> 1.2 | |-- <ESP8266WiFi> 1.0 |-- <noise-c> 0.1.3 | |-- <libsodium> 1.10018.1 |-- <DNSServer> 1.1.1 | |-- <ESP8266WiFi> 1.0 Compiling /data/slimmelezer/.pioenvs/slimmelezer/src/main.cpp.o Linking /data/slimmelezer/.pioenvs/slimmelezer/firmware.elf Building /data/slimmelezer/.pioenvs/slimmelezer/firmware.bin Retrieving maximum program size /data/slimmelezer/.pioenvs/slimmelezer/firmware.elf Checking size /data/slimmelezer/.pioenvs/slimmelezer/firmware.elf RAM: [==== ] 41.1% (used 33668 bytes from 81920 bytes) Flash: [==== ] 44.0% (used 459848 bytes from 1044464 bytes) Creating BIN file "/data/slimmelezer/.pioenvs/slimmelezer/firmware.bin" using "/data/cache/platformio/packages/framework-arduinoespressif8266/bootloaders/eboot/eboot.elf" and "/data/slimmelezer/.pioenvs/slimmelezer/firmware.elf" ========================= [SUCCESS] Took 21.81 seconds ========================= INFO Successfully compiled program. INFO Running: esptool.py --before default_reset --after hard_reset --baud 460800 --port /dev/ttyUSB3 --chip esp8266 write_flash -z 0x0 /data/slimmelezer/.pioenvs/slimmelezer/firmware.bin esptool.py v3.1 Serial port /dev/ttyUSB3 Connecting.... Chip is ESP8266EX Features: WiFi Crystal is 26MHz MAC: c4:5b:be:6c:b3:e7 Uploading stub... Running stub... Stub running... Changing baud rate to 460800 Changed. Configuring flash size... Flash will be erased from 0x00000000 to 0x00071fff... Compressed 464000 bytes to 317488... Wrote 464000 bytes (317488 compressed) at 0x00000000 in 7.4 seconds (effective 499.8 kbit/s)... Hash of data verified. Leaving... Hard resetting via RTS pin... INFO Successfully uploaded program. INFO Starting log output from /dev/ttyUSB3 with baud rate 115200That code is new since yesterday (I changed the accuracy_decimals from 0 to 3). I have this with latest normal ESPHome (2021.9.3) working flawlessly on my test meter.
Do you get between those checksum mismatch also good readings? Those messages aren't necessarily bad, as a meter can spit out corrupt telegrams and than the checksum doesn't match.
Just reinstalled my slimmemeter with the exact code (apart from wif settings) with ESPHome 2021.10.1 and also only getting wifi, uptime and checksum messages. No correct values.
Checksum mismatch
[E][dsmr:154]: !0375
^
Checksum mismatch
[E][dsmr:154]: 1-0:99.97.0(2)(0-0:96.7.19)(180531102609S)(0000000265*s)(190322092303W)(000000!ED7D
^
----
Checksum mismatch
[E][dsmr:154]: !F0�0
^
Incomplete or malformed checksum
[E][dsmr:154]: !FBD8
^
Checksum mismatch
[E][dsmr:154]: !F9D8
^I'm using a ISKRA AM550-ED1.01, SR 5.0
mmakaay
commented
2 years ago There is a different ticket for the CRC errors with 2021.10.0. Check out: https://github.com/esphome/issues/issues/2577
TL;DR:
rx_buffer_size: 1024 for the uart: configmmakaay
commented
2 years ago Closing this issue. A lot of changes have been done to the DSMR component and they have likely solved the existing issues. Also, documentation was written for the DSMR component with tips about getting trustworthy readings.
The problem
Slimmelezer hang or does not update for long periods.
Dsmr is still sending data (when I look at the log of the slimmelezer) but on HA sometimes 5 min or more between 2 updates on screen.
Energy values are not updated correctly values from the add-on are regularly unavailable
Second point: Belgian correction (DSMR Version on 5 chars) still not implemented and had to use this code to enable it to work:
platformio_options: lib_deps: glmnet/Dsmr@0.5 lib_ignore: glmnet/Dsmr@0.3
Thanks for the great work Philippe
Log from esphome INFO slimmelezer.deffet.local: Ping Failed! INFO Disconnected from ESPHome API for slimmelezer.deffet.local WARNING Disconnected from API INFO slimmelezer.deffet.local: Unexpected error while reading incoming messages: 0 bytes read on a total of 116 expected bytes Traceback (most recent call last): File "/usr/local/lib/python3.9/dist-packages/aioesphomeapi/connection.py", line 505, in run_forever await self._run_once() File "/usr/local/lib/python3.9/dist-packages/aioesphomeapi/connection.py", line 480, in _run_once pkt = await self._frame_helper.read_packet() File "/usr/local/lib/python3.9/dist-packages/aioesphomeapi/connection.py", line 219, in read_packet return await self._read_packet_plaintext() File "/usr/local/lib/python3.9/dist-packages/aioesphomeapi/connection.py", line 214, in _read_packet_plaintext raw_msg = await self._reader.readexactly(length_int) File "/usr/lib/python3.9/asyncio/streams.py", line 721, in readexactly raise exceptions.IncompleteReadError(incomplete, n) asyncio.exceptions.IncompleteReadError: 0 bytes read on a total of 116 expected bytes WARNING Can't connect to ESPHome API for slimmelezer.deffet.local: Timeout while connecting to ('192.168.20.174', 6053) INFO Trying to reconnect to slimmelezer.deffet.local in the background INFO Successfully connected to slimmelezer.deffet.local
Which version of ESPHome has the issue?
v2021.9.0
What type of installation are you using?
Home Assistant Add-on
Which version of Home Assistant has the issue?
2021.9.6
What platform are you using?
ESP8266
Board
D1Mini (Slimmelezer)
Component causing the issue
api ?
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Additional information
No response