Closed puterboy closed 9 months ago
Never mind -- this is just due to how numbers are stored in floating point base2 and then converted to base10. Only solution would be to convert to integer types and multiply say by 1 thousand to give equivalent of 3 decimal places. So, for now I will close this bug.
I should note that using integer arithmetic would seemingly be much faster...
The
voltage
,current
,power
, andtotal
fields in the database all have precision of 21 digits to the right of the decimal place while really only 3 seem significant. In fact a lot of them show artifacts of mistaken precision:It seems like the numbers should be rounded to 3 decimal places before storing in the table.