Closed gopherbot closed 9 years ago
Is this a variation of issue #458? fmt: wrong output for format %g https://golang.org/issue/458
I don't believe that it is. issue #458 seems to be saying that %g should have shorter output. I haven't had time to carefully look over the math, but I believe that the rounding isn't working quite right. From comment #2 in issue #458: %g in Go means print as many digits as are necessary for strconv.Atof64 or strconv.Atof32 (depending on the type being printed) to reconstruct exactly the same value. I could be wrong; 0.30000000000000004 could actually be the smallest portion of float64(.1)*3 (appr 0.3000000000000000444089209850062616169452667236328125) required to reconstruct the value.
After running some numbers, I think I might actually be wrong. 0.3 isn't enough to reconstruct 0.3000000000000000444089209850062616169452667236328125, it instead gives 0.299999999999999988897769753748434595763683319091796875 Anything between 0.30000000000000002 and 0.30000000000000007 gives the original value. So basically, the float rounding seems correct from the "as many digits as are necessary for strconv.Atof64 to reconstruct exactly the same value". Still, it seems wrong that .1*3 (.3 using decimal math, but just over .3 when using floating point math) should be printed as 0.30000000000000004, especially since the 0000000000000004 is only part of the number due to floating point imprecision.
Decimal 0.1, used in your examples, is a classic example of a decimal number that can't be exactly represented as a finite binary floating-point number [Goldberg, p4]; we use an approximation. Rounding error is inherent in floating-point calculations. As you have seen, %g rounds with a particular purpose, "as many digits as are necessary for strconv.Atof64 [or strconv.Atof32] to reconstruct exactly the same value"; it works. As issue #458 suggests, use %6g, or something similar, for your purpose. What Every Computer Scientist Should Know About Floating-Point Arithmetic, David Goldberg http://dlc.sun.com/pdf/800-7895/800-7895.pdf
The suggested format of %6g, copied from issue #458, is incorrect; use %.6g for 6 places of precision after the decimal point.
by jeffquiparle:
Attachments: