search

edwig / BCD

Binary Coded Decimal. Research project for use with SQL_NUMERIC_STRUCT datatype
MIT License
12 stars 1 forks source link

bcd number representation or bcd::Exp is incorrect #4

Closed sisirajaya closed 4 years ago

sisirajaya commented 4 years ago

In my experiment I narrowed down to the exponential bcd::Exp function giving wrong answers. bcd exp(0.01) is 1.010050167 Match double ::exp(0.01) is 1.010050167

bcd exp(0.02) is 1.221402758 Do not match with ::exp double ::exp(0.02) is 1.02020134 bcd exp(0.03) is 1.349858808 Do not match with ::exp double ::exp(0.03) is 1.030454534 I think the problem is the bcd representation of .01, .02 and .03.

t = .1 + E-1 10000000 00000000 00000000 00000000 00000000 tt = .01 + E-2 10000000 00000000 00000000 00000000 00000000

t = .2 + E+0 02000000 00000000 00000000 00000000 00000000 tt = .02 + E-1 20000000 00000000 00000000 00000000 00000000

t = .3 + E+0 03000000 00000000 00000000 00000000 00000000 tt = .03 + E-1 30000000 00000000 00000000 00000000 00000000

Here is the code fragment for your convenience:

bcd t = 1 * .1; t.DebugPrint("t = .1"); bcd tt = t / 10.; tt.DebugPrint("tt = .01"); bcd bexp = exp(tt); printf("bcd exp(0.01) is \t%20.10g\n", bexp.AsDouble()); printf("double ::exp(0.01) is \t%20.10g\n", ::exp(.01));

t = 2 * .1; t.DebugPrint("t = .2"); tt = t / 10.; tt.DebugPrint("tt = .02"); bexp = exp(tt); printf("bcd exp(0.02) is \t%20.10g\n", bexp.AsDouble()); printf("double ::exp(0.02) is \t%20.10g\n", ::exp(.02));

t = 3 * .1; t.DebugPrint("t = .3"); tt = t / 10.; tt.DebugPrint("tt = .03"); bexp = exp(tt); printf("bcd exp(0.03) is \t%20.10g\n", bexp.AsDouble()); printf("double ::exp(0.03) is \t%20.10g\n", ::exp(.03));

sisirajaya commented 4 years ago

It is definitely in the number representation. I checked with other functions such as bcd::sin values do not match with std::sin values. Loos like exponent and mantissa computation for decimals are incorrect with exception of .1 is correct. I am not yet up-to-speed with evaluation is them to fix it.

t = .3 + E+0 03000000 00000000 00000000 00000000 00000000 tt = .03 + E-1 30000000 00000000 00000000 00000000 00000000 bcd sin(0.03) is 0.2955202067 double ::sin(0.03) is 0.0299955002

t = 3 * .1; t.DebugPrint("t = .3"); tt = t / 10.; tt.DebugPrint("tt = .03"); bexp = sin(tt); printf("bcd sin(0.03) is \t%20.10g\n", bexp.AsDouble()); printf("double ::sin(0.03) is \t%20.10g\n", ::sin(.03));

sisirajaya commented 4 years ago

It Looks like tt = t/10 is the culprit... when the same number is assigned as a constant 0.02 the answers are correct.

tt= .2/10 --> + E-1 20000000 00000000 00000000 00000000 00000000 (BAD) tt= 0.02 --> + E-1 02000000 00000000 00000000 00000000 00000000 (GOOD)

t = 2 * .1; t.DebugPrint("t = .2"); --> t = .2 + E+0 02000000 00000000 00000000 00000000 00000000 tt = t / 10.; tt.DebugPrint("tt = .02"); --> tt = .02 + E-1 20000000 00000000 00000000 00000000 00000000 tt = 0.02; td = tt.AsDouble(); tt.DebugPrint("tt = .02"); --> tt = .02 + E-1 02000000 00000000 00000000 00000000 00000000 bexp = exp(tt); printf("bcd exp(0.02) is \t%20.10g\n", bexp.AsDouble()); --> 1.020214028 printf("double ::exp(0.02) is \t%20.10g\n", ::exp(.02)); ---> 1.02020134

sisirajaya commented 4 years ago

Possible fix.

In function bcd::SetValueDouble m_exponent was changed to:

// Take care of exponent m_exponent = (short) ::floor(::log10(between));

taking floor of the log gives correct answers to all range of values.