client9 / stringencoders

Fast c-string transformations
MIT License
135 stars 51 forks source link

modp_dtoa() fails for input of 0.95 with precision of 1 #40

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1. call modp_dtoa(1.95, s, 1);
2. call modp_dtoa(0.95, s, 1);
3. call modp_dtoa(0.9999995, s, 6);

What is the expected output? What do you see instead?
1. s = "1.1"
2. s = "1.0"
3. s = "1.0"

What do you see instead?
1. s = "1.10"
2. s = "0.1"
3. s = "0.1"

What version of the product are you using? On what operating system?
Windows 7, TDM-GCC 4.8.1 64-bit with -std=C99 

Please provide any additional information below.
This causes issues for any values where the fractional part is recurring 9's 
followed by a 5 and the precision length would cut off the 5.

Original issue reported on code.google.com by toby.m...@gmail.com on 10 Dec 2014 at 12:55

GoogleCodeExporter commented 9 years ago
case 1. expected output is actually "2.0" of course

Original comment by toby.m...@gmail.com on 10 Dec 2014 at 1:32

GoogleCodeExporter commented 9 years ago
Fix: in file modp_num2a.c 

@ ln 159 insert: 
        frac /= 10;

@ ln 159 change to:  
    } while (count > 0);

@ ln 160: insert:
    // rollover fraction
    if (frac > 0) {
        ++whole;
    }

Original comment by toby.m...@gmail.com on 10 Dec 2014 at 2:09

GoogleCodeExporter commented 9 years ago
oops, file name is actually modp_numtoa.c

Original comment by toby.m...@gmail.com on 10 Dec 2014 at 2:10

jeroen commented 8 years ago

Same problem for modp_dtoa2. Any suggestions for a workaround?

client9 commented 8 years ago

Hello, I'll take a look at this shortly. My apologies for not seeing this sooner. I don't use this library anymore and apparently I was not getting notifications.

PR welcome.

client9 commented 8 years ago

please note glibc and windows use different rounding algorithms.

client9 commented 8 years ago
#include <stdio.h>

int main() {
    printf("%.1f\n", 1.95);
    printf("%.1f\n", 0.95);
    printf("%.6f\n", 0.9999995);
}

glibc

1.9
0.9
1.000000

musl-c

1.9
0.9
1.000000
client9 commented 8 years ago

Due to floating point issues modp_numtoa returns the following:

2.0
1.0
1.000000

which actually matches the standard "round to even" more intuitively.

the problem is that "0.95" isn't actually "95/100" printf looks at the internal representation and figures out that it's a bit less than 95/100 and so rounds down. mod_dtoa doesn't try to make this correct for performance reasons.

I think the original bug is fixed.