jneilliii / OctoPrint-TPLinkSmartplug

105 stars 57 forks source link

[BUG]: Data fields in energy_data database have 21 decimals of precission #347

Closed puterboy closed 9 months ago

puterboy commented 9 months ago

The voltage, current, power, and total fields in the database all have precision of 21 digits to the right of the decimal place while really only 3 seem significant. In fact a lot of them show artifacts of mistaken precision:

CREATE TABLE energy_data(id INTEGER PRIMARY KEY, ip TEXT, timestamp TEXT, current REAL, power REAL, total REAL, voltage REAL);
...
INSERT INTO energy_data VALUES(7,'prusa','2023-12-24 04:36:26.282930',0.13800000000000001155,13.30400000000000027,0.014999999999999999445,123.19799999999999329);
INSERT INTO energy_data VALUES(8,'prusa','2023-12-24 05:56:38.158190',0.0,0.0,0.0240000000000000005,123.59399999999999409);
INSERT INTO energy_data VALUES(9,'prusa','2023-12-24 11:22:20.678944',0.0,0.0,0.0240000000000000005,122.73399999999999466);
INSERT INTO energy_data VALUES(10,'prusa','2023-12-24 11:38:08.332782',0.0,0.0,0.0240000000000000005,120.82999999999999829);
INSERT INTO energy_data VALUES(11,'prusa','2023-12-24 12:02:13.989720',1.3520000000000000906,167.81999999999999318,0.050000000000000002776,121.38299999999999557);

It seems like the numbers should be rounded to 3 decimal places before storing in the table.

puterboy commented 9 months ago

Never mind -- this is just due to how numbers are stored in floating point base2 and then converted to base10. Only solution would be to convert to integer types and multiply say by 1 thousand to give equivalent of 3 decimal places. So, for now I will close this bug.

puterboy commented 9 months ago

I should note that using integer arithmetic would seemingly be much faster...