search

DevO2012 / stringencoders

Automatically exported from code.google.com/p/stringencoders
Other
1 stars 0 forks source link

Is it possible to get higher precision then 9 when using mod_dtoa/mod_dtoa2? #13

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Nick,

I see in the code the following restriction:

    if (prec < 0) {
        prec = 0;
    } else if (prec > 9) {
        /* precision of >= 10 can lead to overflow errors */
        prec = 9;
    }

and

    /* if input is larger than thres_max, revert to exponential */
    const double thres_max = (double)(0x7FFFFFFF);

Could max prec and thres_max be raised? Do I understand correctly that the 
problem is caused by the fact that the code is not using int64_t/uint64_t here:

    int whole = (int) value;
    ...
    uint32_t frac = (uint32_t)(tmp);

Is there any good reason not to switch to 64 bit integers here and allow higher 
prec and thres_max? I'm willing to write a patch but I was just wondering if 
I'm missing something.

Original issue reported on code.google.com by ilya.m...@gmail.com on 10 Jun 2010 at 9:00

GoogleCodeExporter commented 9 years ago 

You are right!

We should use a 64 bit int in the conversion.

I wrote this a long time ago on a 32-bit CPU and wasn't thinking about 64-bit 
ints.

Yes: patch!

--nickg

Original comment by nickgsup...@gmail.com on 12 Jun 2010 at 2:09

GoogleCodeExporter commented 9 years ago

Original comment by nickgsup...@gmail.com on 13 Aug 2012 at 11:15