calc not right: 7x1.15 = 8.049999999999999999?

[ghost]

with the math_expressions library, we type

7*1.15

the calc result was

8.0499999999999

why it not return 8.05

[fkleon]

Hi @cnmade, check out #51 and my answer which I believe describes the same issue.

[ghost]

thanks, i will try

[ghost]

it seems getStringAsFixed can not process very high precision.

nomore@appledeMac-mini ~/tmp $ dart run main.dart
8.05000000000000
8.0499999999999989342
8.049999999999999
nomore@appledeMac-mini ~/tmp $ cat main.dart
void main() {
    double i = 7.0 * 1.15;

    print(i.toStringAsFixed(14));
    print(i.toStringAsFixed(19));
    print(i);

}

[fkleon]

@cnmade You need to select the precision based on your application's requirements. I assume in your case you'd want a low(e) precision- maybe of 3 significant digits - so that the result is rounded correctly:

void main() {
    double i = 7.0 * 1.15;
    print(i.toStringAsFixed(3)); # prints 8.05
}

Note that the original result 8.049999999999999only has 16 digits precision, so when asking for 19 significant digits you're getting a more precise (longer) value as per Dart's implementation of 64 bit double-precision floating point numbers.

Hope that helps!