Umarshal float64 with one too many decimal places

[jakobbrezigar]

Unmarshaling float64 with one too many many decimal places i.e. 0.0080660999999999997 gives me an error.

Simple example:

type ExampleStruct struct {
    Price  float64 `json:"price"`
}
// UnmarshalJSONObject implements gojay's UnmarshalerJSONObject
func (v *ExampleStruct) UnmarshalJSONObject(dec *gojay.Decoder, k string) error {
    switch k {
    case "price":
        return dec.Float64(&v.Price)
    }
    return nil
}
// NKeys returns the number of keys to unmarshal
func (v *ExampleStruct) NKeys() int { return 1 }

func TestGojay(t *testing.T) {
    data:= []byte(`{"price": 0.0080660999999999997}`)
    value := new(ExampleStruct)
    err := gojay.Unmarshal(data, value)
    if err != nil {
        t.Error(err)
    }
}

Suggested solution bellow fixed my issue. Just added one more element to pow10uint64 (10^10) at the end:

var pow10uint64 = [21]uint64{
    0,
    1,
    10,
    100,
    1000,
    10000,
    100000,
    1000000,
    10000000,
    100000000,
    1000000000,
    10000000000,
    100000000000,
    1000000000000,
    10000000000000,
    100000000000000,
    1000000000000000,
    10000000000000000,
    100000000000000000,
    1000000000000000000,
    10000000000000000000,
}

[jakobbrezigar]

I still get errors with these numbers:

data1:= []byte(`{"price": 5620.1400000000003}`)
data2:= []byte(`{"price": 0.00058273999999999999}`)

Consider using slice of float64 for pow10uint64 (var pow10float64 = [24]float64), so I could add bigger divider numbers. Or maybe consider using big.Int?

[francoispqt]

Hi,

Let me check these numbers and try to find a decent solution. I think the standard package rounds these numbers when parsing them, so I think if your intention is to be able to use these numbers with arbitrary precision the best would be to parse them into big.Float from the math/big package.

I'm experimenting to see what we can do, I'll come back to you.

[francoispqt]

So I've decided to truncate these numbers when parsing them. Will push an update later today to master and a new release soon.

[francoispqt]

I've run these benchmarks:

var bigf = []byte(`0.00058273999999999999`)

// BenchmarkBigFloatEncodingJSON decodes a big float with the standard package
func BenchmarkBigFloatEncodingJSON(b *testing.B) {
    b.ReportAllocs()
    for n := 0; n < b.N; n++ {
        var f float64
        var _ = json.Unmarshal(bigf, &f)
    }
}

// BenchmarkBigFloatGojay decodes a big float with gojay
func BenchmarkBigFloatGojay(b *testing.B) {
    b.ReportAllocs()
    for n := 0; n < b.N; n++ {
        var f float64
        var _ = gojay.Unmarshal(bigf, &f)
    }
}

And the results are the following:

--- encoding/json
goos: darwin
goarch: amd64
pkg: github.com/francoispqt/gojay/benchmarks/decoder
BenchmarkBigFloatEncodingJSON-8      2000000           627 ns/op         328 B/op          3 allocs/op
PASS
ok      github.com/francoispqt/gojay/benchmarks/decoder 1.929s
Success: Benchmarks passed.

--- gojay
goos: darwin
goarch: amd64
pkg: github.com/francoispqt/gojay/benchmarks/decoder
BenchmarkBigFloatGojay-8    10000000           177 ns/op           8 B/op          1 allocs/op
PASS
ok      github.com/francoispqt/gojay/benchmarks/decoder 1.989s
Success: Benchmarks passed.

[francoispqt]

Please check with the current master and close the issue. I will add decoding to big.Float and big.Int, you could already do it by first decoding to a gojay.EmbeddedJSON which is basically a []byte and then use big.ParseFloat.

[francoispqt]

Not getting any answer so I'm closing. Just reopen if it is not fixed. Thanks!