search

serso / android-calculatorpp

Android Calculator
421 stars 201 forks source link

Floating point division rounding #177

Open LeMinaw opened 7 years ago

LeMinaw commented 7 years ago

When performing 2.45/0.35, the calculator returns 7.0000000000000001. The expected result is obviously 7.

Android v5.1 CPP v2.2.3

idib commented 7 years ago

a same problem 10.53+25.12+62.3+68.72+65.902 => 413.2000000000001

68.72+65.902 => 269.2000000000001 (68.7+65.90)2 => 269.2000000000001 68.7+65.90 => 134.6 68.72+65.90 => 203.3 68.7+65.90*2 => 200.5

calculatorpp V2.2.3 Android v6.0

acshef commented 7 years ago

Reduced test case:

8.7 - 1 = 7.699999999999999

Android v6.0.1 CPP v2.2.3