Xlythe / Calculator

208 stars 91 forks source link

Only 6 decimal places of precision #46

Open tagallow opened 8 years ago

tagallow commented 8 years ago

If you calculate an irrational number, such as pi or the square root of 2, the calculator only gives you 6 decimal places of precision. This probably isn't a huge deal for casual use, but it seems like a very arbitrary limitation to have in 2016. Here is a demonstration of this app vs the stock android app, using the square root of 2.

Using the Xlythe app first:

Calculate SQRT(2) and hit [=] xlythe1

The UI only shows 6 decimal places. To test if this is just a UI issue or a calculation issue, we can multiply that result by SQRT(2) and see what the result is:

xlythe2

2.000001

Now the same process with the stock android app in Android 7.0:

stock1

16 decimal places, lets see how it carries over.

stock2

2, the correct answer