search

ericlagergren / decimal

A high-performance, arbitrary-precision, floating-point decimal library.
https://godoc.org/github.com/ericlagergren/decimal
BSD 3-Clause "New" or "Revised" License
518 stars 61 forks source link

Float64 conversion #174

Open dcullender-cb opened 2 years ago

dcullender-cb commented 2 years ago

When converting to float64 using Float64(), the value can differ from the input string while shopspring/decimal seems to produce the correct result. for example

image

Appreciate that the value cannot fit into a float64 as per ericOk but curious why the shopspring value matches?

ericlagergren commented 2 years ago

This is because shopspring converts their decimal to a big.Rat, then converts the Rat to a float64. My library converts as-is using float64.

The upside to shopspring’s method is that it won’t round worse than the decimal’s current precision during the conversion. The downside is that converting to a rational number is expensive: you end up with huge numerators and denominators. Plus, you have to rescale the decimal to create the rational number.

The upside to my method is it’s fast and allocation-free (for small floats, that is). The downside is you occasionally have rounding issues.

There might be a better way of performing the conversion (more bit twiddling instead of float division/multiplication), but I haven’t looked too closely at it. I’m not entirely sure why people are using the Float64 method—it seems to defeat the purpose of using a decimal library.