serso / android-calculatorpp

Android Calculator
421 stars 204 forks source link

IEEE754 Edge Cases (aka 0/0 should be NaN but resolves to 0) #170

Open wischi-chr opened 8 years ago

wischi-chr commented 8 years ago

There are a number of edge cases which are wrong. The java datatype "double" implements IEEE 754 and Calculator++ (mostly) follows the same rules (see Infinity and NaN).

But some calculations are wrong, here is a list:

Expression Result (commit 3c1e3bb) Should be (according to IEEE 754)
0 / 0 0 NaN
0.0 / 0.0 1 NaN
0.0 / 0 NaN NaN
0 ^ 0 1 1 (IMO* this should be NaN too :smile: )
∞ * 0 0 NaN
∞ / ∞ 1 NaN
∞ - ∞ 0 NaN
NaN - NaN 0 NaN
NaN / NaN 1 NaN

* and Wolfram Alpha agrees with me ^^ (http://www.wolframalpha.com/input/?i=0%5E0)

I think the problem is, that those expressions are reduced before they are calculated and Infinity and NaN are processed like normal variables/numbers (which they are not).

Java (double datatype) would calculate those expressions correctly see here: https://ideone.com/Oy1Qhz

import java.util.*;
import java.lang.*;
import java.io.*;

class DemoIEEE754
{
    public static void main (String[] args) throws java.lang.Exception
    {
        System.out.println("0/0=" + (0./0.));
        System.out.println("0^0=" + Math.pow(0.,0.));

        System.out.println("∞*0=" + (Double.POSITIVE_INFINITY*0.));
        System.out.println("∞/∞=" + (Double.POSITIVE_INFINITY/Double.POSITIVE_INFINITY));
        System.out.println("∞-∞=" + (Double.POSITIVE_INFINITY-Double.POSITIVE_INFINITY));

        System.out.println("NaN-NaN=" + (Double.NaN-Double.NaN));
        System.out.println("NaN/NaN=" + (Double.NaN/Double.NaN));
    }
}

Every operation with NaN should always result in NaN even with variables. There is no way an undefinied expression (NaN) could lead to an non-NaN result.

wischi-chr commented 7 years ago

Any chance this will be fixed?