arendst / Tasmota

Alternative firmware for ESP8266 and ESP32 based devices with easy configuration using webUI, OTA updates, automation using timers or rules, expandability and entirely local control over MQTT, HTTP, Serial or KNX. Full documentation at
https://tasmota.github.io/docs
GNU General Public License v3.0
22.09k stars 4.79k forks source link

Delay/Skip first tele: Smart Meter first message full of zeros #11053

Closed nagyrobi closed 3 years ago

nagyrobi commented 3 years ago

PROBLEM DESCRIPTION

After reboot or power cycle, the first tele message on MQTT is sent with all the values at zero. Supposedly because Smart Meter interface didn't receive values from the meter yet. P1 serial meters usually send values at every 10 seconds. Tasmota boots and sends the first tele faster, and the zeros disturb the accuracy on the database.

Seems like I'm not the only one experiencing this:

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

- [x] Provide the output of this command: `Status 0`:
```lua
00:14:05.064 CMD: STATUS 0
00:14:05.072 MQT: stat/bal/STATUS = {"Status":{"Module":18,"DeviceName":"Bal (Normál)","FriendlyName":["Bal (Normál)"],"Topic":"bal","ButtonTopic":"0","Power":0,"PowerOnState":3,"LedState":1,"LedMask":"FFFF","SaveData":1,"SaveState":1,"SwitchTopic":"0","SwitchMode":[0,0,0,0,0,0,0,0],"ButtonRetain":0,"SwitchRetain":0,"SensorRetain":0,"PowerRetain":0}}
00:14:05.081 MQT: stat/bal/STATUS1 = {"StatusPRM":{"Baudrate":115200,"SerialConfig":"8N1","GroupTopic":"tasmotas","OtaUrl":"http://ota.tasmota.com/tasmota/release/tasmota.bin.gz","RestartReason":"Software/System restart","Uptime":"0T00:05:00","StartupUTC":"2021-02-19T23:09:05","Sleep":50,"CfgHolder":4617,"BootCount":5,"BCResetTime":"2021-02-19T23:31:47","SaveCount":11,"SaveAddress":"3FB000"}}
00:14:05.087 MQT: stat/bal/STATUS2 = {"StatusFWR":{"Version":"9.2.0.7(tasmota)","BuildDateTime":"2021-02-19T22:50:42","Boot":31,"Core":"2_7_4_9","SDK":"2.2.2-dev(38a443e)","CpuFrequency":80,"Hardware":"ESP8266EX","CR":"461/699"}}
00:14:05.094 MQT: stat/bal/STATUS3 = {"StatusLOG":{"SerialLog":0,"WebLog":2,"MqttLog":0,"SysLog":0,"LogHost":"","LogPort":514,"SSId":["AA",""],"TelePeriod":60,"Resolution":"558180C0","SetOption":["00008009","2805C8000100060000005A0A000000000000","000000C0","00006000","00000000"]}}
00:14:05.105 MQT: stat/bal/STATUS4 = {"StatusMEM":{"ProgramSize":654,"Free":1392,"Heap":11,"ProgramFlashSize":4096,"FlashSize":4096,"FlashChipId":"1640D8","FlashFrequency":40,"FlashMode":3,"Features":["00000809","87DAC787","043E8001","000000CF","010013C0","C000F989","00024004","00201000","00000000"],"Drivers":"1,2,3,4,5,6,7,8,9,10,12,16,18,19,20,21,22,24,26,27,29,30,35,37,41,45,50","Sensors":"1,2,3,4,5,6,53"}}
00:14:05.115 MQT: stat/bal/STATUS5 = {"StatusNET":{"Hostname":"bal-3514","IPAddress":"192.168.1.48","Gateway":"192.168.1.254","Subnetmask":"255.255.255.0","DNSServer":"192.168.1.254","Mac":"8C:AA:B5:7C:CD:BA","Webserver":2,"WifiConfig":4,"WifiPower":17.0}}
00:14:05.121 MQT: stat/bal/STATUS6 = {"StatusMQT":{"MqttHost":"192.168.1.250","MqttPort":1883,"MqttClientMask":"DVES_%06X","MqttClient":"DVES_7CCDBA","MqttUser":"hamqt","MqttCount":1,"MAX_PACKET_SIZE":1200,"KEEPALIVE":30}}
00:14:05.129 MQT: stat/bal/STATUS7 = {"StatusTIM":{"UTC":"2021-02-19T23:14:05","Local":"2021-02-20T00:14:05","StartDST":"2021-03-28T02:00:00","EndDST":"2021-10-31T03:00:00","Timezone":"+01:00","Sunrise":"06:39","Sunset":"17:13"}}
00:14:05.138 MQT: stat/bal/STATUS10 = {"StatusSNS":{"Time":"2021-02-20T00:14:05","N":{"enrg_imp":41.157,"enrg_exp":3.762,"telj_imp":1.241,"telj_exp":0.000,"fesz_l1":228.4,"fesz_l2":230.2,"fesz_l3":232.8,"freq":49.94,"enrg_imp_med":0.568,"enrg_imp_med":19.463,"havi":"210201000000W"}}}
00:14:05.146 MQT: stat/bal/STATUS11 = {"StatusSTS":{"Time":"2021-02-20T00:14:05","Uptime":"0T00:05:00","UptimeSec":300,"Heap":11,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":1,"Wifi":{"AP":1,"SSId":"AA","BSSId":"F0:9F:C2:A4:3B:1C","Channel":3,"RSSI":74,"Signal":-63,"LinkCount":1,"Downtime":"0T00:00:07"}}}

TO REPRODUCE

Reboot the module.

EXPECTED BEHAVIOUR

Do not send a tele until values are filled.

SCREENSHOTS

None.

ADDITIONAL CONTEXT

The script:


>D  
>B  
->sensor53 r
>M 1  
+1,3,o,16,115200,N,1
1,1-0:1.8.0(@1,Energia import,kWh,enrg_imp,3
1,1-0:2.8.1(@1,Energia export,kWh,enrg_exp,3
1,0:1.7.0(@1,Teljesítmény import,kW,telj_imp,3
1,0:2.7.0(@1,Teljesítmény export,kW,telj_exp,3
1,32.7.0(@1,L1 Feszültség,V,fesz_l1,1
1,52.7.0(@1,L2 Feszültség,V,fesz_l2,1
1,72.7.0(@1,L3 Feszültség,V,fesz_l3,1
1,0:14.7.0(@1,Frekvencia,Hz,freq,2
1,0:3.8.0(@1,Energia meddő import,kvarh,enrg_imp_med,3
1,0:4.8.0(@1,Energia meddő export,kvarh,enrg_imp_med,3
1,0:98.1.0(@#),Havi adat,,havi,0
nagyrobi commented 3 years ago

I tried various workarounds like setting "tele_period": 0 from config.sys at boot, and add delay and teleperiod restore to the script:


>BS
delay(10000)
tper=30

But no success. A tele is sent immediately after boot with lots of zeros.

arendst commented 3 years ago

The smart meter code has to be written in that it not responds to the teleperiod request if it has not yet received valid data.

Over to the smartmeter code writer @gemu2015 ....

nagyrobi commented 3 years ago

Thanks for this. It's not perfect yet, it only works once. So after I freshly upgraded the firmware to a build containing this patch, at the first boot it indeed skipped the first tele. But after subsequent restarts from the web gui, a null mqtt is sent after boot... :(

gemu2015 commented 3 years ago

meanwhile I rewired my sml test system and if no serial data comes in no MQTT is generated. i made several restarts of sml and rebooting and could not observe a null mqtt. please test in console with sensor53 r and check if a null mqtt is generated. the valid flags are always reset on restart and once a serial datagram is received the valid flag is set and only then mqtt is sent.

i also did implement a global sml mqtt disable where you can disable sml mqtt from scripts. the script variable is smlj you may read and write it. a 1 enables mqtt from script, a zero disables it.

i also implemented the bracket skipping. here is a new version. now that i can check with my simulator i found a bug in the previous version.

please test

Archiv.zip

nagyrobi commented 3 years ago

Unfortunately still not good:

0:00:00.008 UFS: FlashFS mounted with 1984 kB free
00:00:00.056 CFG: Loaded from File, Count 6
00:00:00.062 QPC: Count 1
00:00:00.068 Project tasmota Bal (Normál) Version 9.3.0.1(tasmota)-2_7_4_9(2021-02-20T20:41:15)
00:00:00.069 UFILESYSTEM OK!
00:00:00.075 Script: nv=0, tv=0, vns=0, ram=16
00:00:00.085 SNS: Hardware Serial
00:00:04.475 WIF: Connecting to AP1 ZACUT Channel 6 BSSId F0:9F:C2:A4:1A:28 in mode 11n as elmu_bal-3514...
00:00:05.801 WIF: Connected
00:00:06.004 HTP: Web server active on elmu_bal-3514 with IP address XXXXXXXX
00:00:06.503 QPC: Reset
20:45:16.013 MQT: Attempting connection...
20:45:16.069 MQT: Connected
20:45:16.072 MQT: tele/elmu_bal/LWT = Online (retained)
20:45:16.075 MQT: cmnd/elmu_bal/POWER = 
20:45:16.083 MQT: tele/elmu_bal/INFO1 = {"Module":"Generic","Version":"9.3.0.1(tasmota)","FallbackTopic":"cmnd/DVES_7CCDBA_fb/","GroupTopic":"cmnd/tasmotas/"}
20:45:16.086 MQT: tele/elmu_bal/INFO2 = {"WebServerMode":"Admin","Hostname":"bal-3514","IPAddress":"XXXXXXXX"}
20:45:16.089 MQT: tele/elmu_bal/INFO3 = {"RestartReason":"Software/System restart"}
20:45:20.487 MQT: tele/elmu_bal/STATE = {"Time":"2021-02-20T20:45:20","Uptime":"0T00:00:13","UptimeSec":13,"Heap":14,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":19,"MqttCount":1,"Wifi":{"AP":1,"SSId":"AAA","BSSId":"F0:9F:C2:A4:1A:28","Channel":6,"RSSI":50,"Signal":-75,"LinkCount":1,"Downtime":"0T00:00:07"}}
20:45:20.494 MQT: tele/elmu_bal/SENSOR = {"Time":"2021-02-20T20:45:20","N":{"enrg_imp":0.000,"enrg_exp":0.000,"telj_imp":0.000,"telj_exp":0.000,"fesz_l1":0.0,"fesz_l2":0.0,"fesz_l3":0.0,"freq":0.00,"enrg_imp_med":0.000,"enrg_imp_med":0.000}}

Maybe it's not too good to rely solely on serial dataflow detection. I assume in my case that if reboot happens in the middle of the data being pushed, only part of the information will be decoded, rest of it will remain zero. For instance I noticed several times that the last values were populated, the first ones were still at zero.

nagyrobi commented 3 years ago

Can you suggest how to use smlj in my script above? I imagine something like set it to 0 at boot. Then wait 2 subsequent serial data pushes (because the first one may be incomplete) and then set smlj to 1.

Tried this, without success:


>D  
>B
smlj 0
->sensor53 r
>BS
delay(15000)
smlj 1
nagyrobi commented 3 years ago

Solution: https://github.com/arendst/Tasmota/discussions/11052#discussioncomment-389123 (for those directed here by a search engine)