search

nagyistoce / stringencoders

Automatically exported from code.google.com/p/stringencoders
Other
0 stars 0 forks source link

modp_dtoa() fails for input of 0.95 with precision of 1 #40

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. call modp_dtoa(1.95, s, 1);
2. call modp_dtoa(0.95, s, 1);
3. call modp_dtoa(0.9999995, s, 6);

What is the expected output? What do you see instead?
1. s = "1.1"
2. s = "1.0"
3. s = "1.0"

What do you see instead?
1. s = "1.10"
2. s = "0.1"
3. s = "0.1"

What version of the product are you using? On what operating system?
Windows 7, TDM-GCC 4.8.1 64-bit with -std=C99 

Please provide any additional information below.
This causes issues for any values where the fractional part is recurring 9's 
followed by a 5 and the precision length would cut off the 5.

Original issue reported on code.google.com by toby.m...@gmail.com on 10 Dec 2014 at 12:55

GoogleCodeExporter commented 9 years ago 
case 1. expected output is actually "2.0" of course

Original comment by toby.m...@gmail.com on 10 Dec 2014 at 1:32

GoogleCodeExporter commented 9 years ago 
Fix: in file modp_num2a.c 

@ ln 159 insert: 
        frac /= 10;

@ ln 159 change to:  
    } while (count > 0);

@ ln 160: insert:
    // rollover fraction
    if (frac > 0) {
        ++whole;
    }

Original comment by toby.m...@gmail.com on 10 Dec 2014 at 2:09

GoogleCodeExporter commented 9 years ago 
oops, file name is actually modp_numtoa.c

Original comment by toby.m...@gmail.com on 10 Dec 2014 at 2:10