search

icasdri / Mather

A powerful expression-based calculator, unit converter, and general computation engine for Android
GNU General Public License v3.0
58 stars 7 forks source link

Floating Point Rounding #2

Closed paride closed 7 years ago

paride commented 8 years ago

I own a Nexus 6 with stock android, so it's a very standard environment. I found out that it makes some calculation errors:

1/3 = 0.3333333333333333 (ok) 1 + 1/3 = 1.3333333333333333 (ok) 2 + 1/3 = 2.3333333333333335 (wtf?!) 3 + 1/3 = 3.3333333333333335 (wtf?!) 4 + 1/3 = 4.3333333333333333 (ok again until 7+1/3) 8 + 1/3 = 7.3333333333333334 (?!, goes like this until 15+1/3) 16+1/3 = 7.3333333333333338 (and with this I got bored).

This looks bad. I did a quick test with the Calculator app from Google and the problem is not present there.

icasdri commented 8 years ago

This is an unfortunate side-effect of the fact that computers operate in binary. Any calculation involving floating point numbers have the floating point rounding problem, as it were. See http://floating-point-gui.de/ for more information if you're interested. Try typing the following expression into Google search: 399999999999999-399999999999998. You'll be surprised at the result.

Indeed the real problem here is not that the calculations are wrong, it's the fact that Math.js shows too many (all the digits it knows of in fact) digits by default. I suspect the official Calculator app does not show this many digits. Thus, a solution would involve rounding off Math.js decimal answers.

paride commented 8 years ago

I see, the stock Calculator app indeed trims the result to show less decimal digits. I often use this calculator (apcalc package on Debian):

http://www.isthe.com/chongo/tech/comp/calc/

which is arbitrary precision.

icasdri commented 8 years ago

Since Mather relies quite extensively on Math.js for calculations and expression parsing, we'll probably need to stick with floating-point numbers and just displaying them in a nicer fashion. Thanks for the info though.

A quick off-topic question: have you tried unit conversions and such. Just as a quick survey about the importance of #3.

paride commented 8 years ago

It seems that arbitrary precision calculations will be implemented in math.js: https://github.com/josdejong/mathjs/issues/694.

icasdri commented 8 years ago

If that happens, we'll inherit that feature :)

icasdri commented 8 years ago

Arbitrary precision fraction support appears to already be available via a configuration option, see http://mathjs.org/docs/core/configuration.html. We will make this available as an option once Settings (#1) is implemented.