search

eyalroz / printf

Tiny, fast(ish), self-contained, fully loaded printf, sprinf etc. implementation; particularly useful in embedded systems.
MIT License
419 stars 52 forks source link

Precision not honoured in large double #151

Closed josesimoes closed 1 year ago

josesimoes commented 1 year ago

Hi,

Trying to output this double 1.7976931348623157E+308 with this formatting string %0.15G and I'm getting: 1.79769313486235E+308 wich is "dropping" the last digit (there are only 14).

I've trace this down to this line: https://github.com/eyalroz/printf/blob/c2e239dff18152e0617768cde65a502b13c8417f/src/printf/printf.c#L928

it's subtracting 1 to the requested precision, thus the result.

eyalroz commented 1 year ago

This is exactly what's mandated by the standard library: For %g, the number of decimal digits past the point is p-1 for a precision of p. See here.

josesimoes commented 1 year ago

Got it. Thanks.

Despite of that there is still something not correct on the library...

If I increase precision to %0.16G I get 1.797693134862348E+308. With %0.17G I get 1.7976931348623476E+308.

According to onlinegdb the correct output should be:

0.16G >>>> 1.797693134862316E+308 0.17G >>>> 1.7976931348623157E+308

Seems that the rounding is not accurate. Can you please confirm?

eyalroz commented 1 year ago

First, it's true the the library's last and perhaps second-last digit precision is imperfect. Improving it is not trivial and may involve a lot of code. That being said - double-precision numbers don't have 16 precision digits. Anyway, different bug.