ENERGY_Total gets truncated to 3 decimals on every date rollover (i.e. at midnight)

MaxRower commented 7 months ago

PROBLEM DESCRIPTION

When I view the graph of ENGERY_Total of my various NOUS A1T plugs, I can see a small drop on every date rollover at midnight. After examining the data in influxdb, I can see, the reported value of ENERGY_Total get's truncated to 3 decimal digits on every date rollover. I have configured to use 5 decimal digits.

REQUESTED INFORMATION

Make sure your have performed every step and checked the applicable boxes before submitting your issue. Thank you!

[x] Read the Contributing Guide and Policy and the Code of Conduct
[x] Searched the problem in issues
[x] Searched the problem in discussions
[x] Searched the problem in the docs
[x] Searched the problem in the chat
[x] Device used (e.g., Sonoff Basic): NOUS A1T
[x] Tasmota binary firmware version number used: 13.3.0
- [x] Pre-compiled
- [ ] Self-compiled
[x] Flashing tools used: OTA

[x] Provide the output of command: Backlog Template; Module; GPIO 255:


Configuration output here:
12:56:51.977 MQT: stat/tasmota_17E79B/RESULT = {"NAME":"NOUS A1T","GPIO":[32,0,0,0,2720,2656,0,0,2624,320,224,0,0,0],"FLAG":0,"BASE":49}
12:56:52.186 MQT: stat/tasmota_17E79B/RESULT = {"Module":{"0":"NOUS A1T"}}
12:56:52.393 MQT: stat/tasmota_17E79B/RESULT = {"GPIO0":{"32":"Button1"},"GPIO1":{"0":"None"},"GPIO2":{"0":"None"},"GPIO3":{"0":"None"},"GPIO4":{"2720":"BL0937 CF"},"GPIO5":{"2656":"HLWBL CF1"},"GPIO9":{"0":"None"},"GPIO10":{"0":"None"},"GPIO12":{"2624":"HLWBL SEL_i"},"GPIO13":{"320":"Led_i1"},"GPIO14":{"224":"Relay1"},"GPIO15":{"0":"None"},"GPIO16":{"0":"None"},"GPIO17":{"0":"None"}}

- [ ] If using rules, provide the output of this command: `Backlog Rule1; Rule2; Rule3`:
```lua
  Rules output here:

[x] Provide the output of this command: Status 0:


STATUS 0 output here:
12:57:52.101 CMD: Status 0
12:57:52.109 MQT: stat/tasmota_17E79B/STATUS = {"Status":{"Module":0,"DeviceName":"Keller Backup","FriendlyName":["Keller Backup"],"Topic":"tasmota_17E79B","ButtonTopic":"0","Power":0,"PowerOnState":0,"LedState":1,"LedMask":"FFFF","SaveData":1,"SaveState":1,"SwitchTopic":"0","SwitchMode":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"ButtonRetain":0,"SwitchRetain":0,"SensorRetain":0,"PowerRetain":0,"InfoRetain":0,"StateRetain":0,"StatusRetain":0}}
12:57:52.115 MQT: stat/tasmota_17E79B/STATUS1 = {"StatusPRM":{"Baudrate":115200,"SerialConfig":"8N1","GroupTopic":"tasmotas","OtaUrl":"http://ota.tasmota.com/tasmota/release/tasmota.bin.gz","RestartReason":"Software/System restart","Uptime":"11T00:17:51","StartupUTC":"2024-01-25T11:40:01","Sleep":50,"CfgHolder":4617,"BootCount":17,"BCResetTime":"2023-03-17T23:12:03","SaveCount":1617,"SaveAddress":"FB000"}}
12:57:52.120 MQT: stat/tasmota_17E79B/STATUS2 = {"StatusFWR":{"Version":"13.3.0(tasmota)","BuildDateTime":"2023-12-12T14:29:05","Boot":31,"Core":"2_7_4_9","SDK":"2.2.2-dev(38a443e)","CpuFrequency":80,"Hardware":"ESP8266EX","CR":"361/699"}}
12:57:52.124 MQT: stat/tasmota_17E79B/STATUS3 = {"StatusLOG":{"SerialLog":2,"WebLog":2,"MqttLog":0,"SysLog":0,"LogHost":"","LogPort":514,"SSId":["IoT",""],"TelePeriod":10,"Resolution":"568B80C0","SetOption":["00208009","2805C80001000600003C5A0A192800000000","00000081","00006000","00004000","00000000"]}}
12:57:52.138 MQT: stat/tasmota_17E79B/STATUS4 = {"StatusMEM":{"ProgramSize":637,"Free":364,"Heap":20,"ProgramFlashSize":1024,"FlashSize":1024,"FlashChipId":"1460C4","FlashFrequency":40,"FlashMode":"DOUT","Features":["0809","8F9AC787","04368001","000000CF","010013C0","C000F981","00004004","00001000","54000020","00000080","00000000"],"Drivers":"1,2,3,4,5,6,7,8,9,10,12,16,18,19,20,21,22,24,26,27,29,30,35,37,45,62,68","Sensors":"1,2,3,4,5,6","I2CDriver":"7"}}
12:57:52.144 MQT: stat/tasmota_17E79B/STATUS5 = {"StatusNET":{"Hostname":"tasmota-17E79B-1947","IPAddress":"10.96.1.243","Gateway":"10.96.1.1","Subnetmask":"255.255.255.0","DNSServer1":"10.96.1.1","DNSServer2":"0.0.0.0","Mac":"48:55:19:17:E7:9B","Webserver":2,"HTTP_API":1,"WifiConfig":4,"WifiPower":17.0}}
12:57:52.149 MQT: stat/tasmota_17E79B/STATUS6 = {"StatusMQT":{"MqttHost":"********","MqttPort":1883,"MqttClientMask":"DVES_%06X","MqttClient":"DVES_17E79B","MqttUser":"DVES_USER","MqttCount":1814,"MAX_PACKET_SIZE":1200,"KEEPALIVE":30,"SOCKET_TIMEOUT":4}}
12:57:52.155 MQT: stat/tasmota_17E79B/STATUS7 = {"StatusTIM":{"UTC":"2024-02-05T11:57:52","Local":"2024-02-05T12:57:52","StartDST":"2024-03-31T02:00:00","EndDST":"2024-10-27T03:00:00","Timezone":99,"Sunrise":"08:15","Sunset":"17:52"}}
12:57:52.159 MQT: stat/tasmota_17E79B/STATUS9 = {"StatusPTH":{"PowerDelta":[0,0,0],"PowerLow":0,"PowerHigh":0,"VoltageLow":0,"VoltageHigh":0,"CurrentLow":0,"CurrentHigh":0}}
12:57:52.173 MQT: stat/tasmota_17E79B/STATUS10 = {"StatusSNS":{"Time":"2024-02-05T12:57:52","ENERGY":{"TotalStartTime":"2024-01-25T12:44:55","Total":3.11794,"Yesterday":0.41687,"Today":0.08394,"Power":0.0,"ApparentPower":0.0,"ReactivePower":0.0,"Factor":0.00,"Voltage":231.3,"Current":0.000}}}
12:57:52.180 MQT: stat/tasmota_17E79B/STATUS11 = {"StatusSTS":{"Time":"2024-02-05T12:57:52","Uptime":"11T00:17:51","UptimeSec":951471,"Heap":20,"SleepMode":"Dynamic","Sleep":50,"LoadAvg":24,"MqttCount":1814,"POWER":"OFF","Wifi":{"AP":1,"SSId":"IoT","BSSId":"02:0C:42:3A:D9:3B","Channel":11,"Mode":"11n","RSSI":32,"Signal":-84,"LinkCount":678,"Downtime":"1T14:06:17"}}}

- [ ] Set `weblog` to 4 and then, when you experience your issue, provide the output of the Console log:
```lua
  Console output here:

TO REPRODUCE

Steps to reproduce the behavior: set EnergyRes 5 and wait for a date rollover

EXPECTED BEHAVIOUR

A clear and concise description of what you expected to happen. The value of ENERGY_Total should increase constantly, without any drops in between

SCREENSHOTS

If applicable, add screenshots to help explain your problem. grafik Timestamps in influxdb seem to be UTC, so they are one hour off from local time.

ADDITIONAL CONTEXT

Add any other context about the problem here.

(Please, remember to close the issue when the problem has been addressed)

MaxRower commented 7 months ago

Well, I checked the values of another plug, it only decreases there, but not truncated to exactly 3 decimals. So maybe some sort of binary truncation actually. grafik

And that produces some weird results in grafana: grafik

sfromis commented 7 months ago

I'm also seeing occasional small drops in "Total" energy at midnight, on some devices, including one with a very stable consumption pattern, without the total being over the about 7 digits of available resolution with single precision floating point.

{"Time":"2024-02-04T23:56:32.323+01:00","ENERGY":{"TotalStartTime":"2022-11-08T17:55:27","Total":44.79066,"Yesterday":0.12747,"Today":0.12666,"Period":0.440,"Power":4.900,"ApparentPower":14.692,"ReactivePower":13.900,"Factor":0.33,"Voltage":233.200,"Current":0.063}}
{"Time":"2024-02-05T00:01:32.326+01:00","ENERGY":{"TotalStartTime":"2022-11-08T17:55:27","Total":44.79013,"Yesterday":0.12696,"Today":0.00013,"Period":0.430,"Power":5.100,"ApparentPower":14.692,"ReactivePower":13.800,"Factor":0.35,"Voltage":233.200,"Current":0.063}}

arendst commented 7 months ago

As designed.

On ESP8266 the total energy is stored in an uint32_t for legacy reasons with max three decimals to allow a max value of +/-2147483.647 kWh On ESP32 total energy stored as a float to allow a max value of +/-262143.99 kWh

stefanbode commented 7 months ago

I assume that not the limited decimals are the problem, but that on midnight when storing the daily consumption into total there is a jump.

stefanbode commented 7 months ago

I was able to find this also on my devices but only on the ones with super low power consumption. If there s e.g. 50w, you see nothing.

sfromis commented 7 months ago

With Default TelePeriod of 5 minutes, you'd not see a fall in first payload after midnight, if consumption since midnight is more than the lost decimals. I suppose that you'd still miss up to 1 Wh in the total.

sfromis commented 7 months ago

What I gather is that the internal "Total" value always only has 3 decimals (1 Wh resolution), and when the "Total" value appearing in TelePeriod payloads can have more than 3 decimals, this is due to it being the value updated at last midnight plus the "Today" value, thus allowing extra decimals from the "Today" accumulation since midnight.

This has not been bothering me, as I hardly use the "Total" value, due to this getting low on decimals when total consuption gets into the MWh range (which would be 9 digits when wanting 5 decimals), not allowing full resolution in a float. Hence my plots are based on differences in the "Today" value accumulated over time, without lowering decimals.

github-actions[bot] commented 6 months ago

This issue has been automatically marked as stale because it hasn't any activity in last few weeks. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 6 months ago

This issue was automatically closed because of being stale. Feel free to open a new one if you still experience this problem.

BigAl66 commented 2 months ago

Did anybody found a solution? I have the same problem here. Every modnight some of my NOUS A1Z have the dame issue. Unfortunately Home Assistant cannot be configured to ignore negative values... (I didn't find at least an option for that...).

arendst / Tasmota