search

Quuxplusone / LLVMBugzillaTest

0 stars 0 forks source link

std::numeric_limits<long double>::max_digits10 massively wrong for ppc #26445

Open Quuxplusone opened 8 years ago

Quuxplusone commented 8 years ago
Bugzilla Link PR26446
Status NEW
Importance P normal
Reported by Richard Smith (richard-llvm@metafoo.co.uk)
Reported on 2016-02-02 20:02:11 -0800
Last modified on 2016-02-03 21:23:09 -0800
Version unspecified
Hardware PC Linux
CC hfinkel@anl.gov, hstong@ca.ibm.com, kit.barton@gmail.com, llvm-bugs@lists.llvm.org, mclow.lists@gmail.com, nemanja.i.ibm@gmail.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also 
libc++ thinks that std::numeric_limits<long double>::max_digits10 is 33 for
ppc64-linux. However, both DBL_MAX + DBL_DENORM_MIN and DBL_MAX +
DBL_DENORM_MIN*2 can be exactly represented by a PPC double double, and you
need around 633 decimal digits to distinguish those two values.
Quuxplusone commented 8 years ago 
The precision of PPCDoubleDouble being variable, the guidance given by the C
Standard for <float.h> is much less useful.

For the purposes of determining LDBL_EPSILON, the definition given in prose
would set "p" to be 1075; however, IBM's compilers on AIX have traditionally
used 106 for "p" for LDBL_EPSILON (since that is a more "conservative" value of
p in some contexts).

It appears that the current value of 33 is derived from p = 106.
For LDBL_DECIMAL_DIG, the definition in prose would suggest that "p" is 2098.

It seems clear that the most useful value for "p" differs between the various
constants (indeed, for LDBL_MAX, the definition in prose would suggest a value
of "p" that is not an integer); however, I am not certain that, for each
particular constant, there is necessarily a value which is clearly "most
useful".