json-iterator / go

A high-performance 100% compatible drop-in replacement of "encoding/json"
http://jsoniter.com/migrate-from-go-std.html
MIT License
13.43k stars 1.03k forks source link

[float string]different result vs origin after call Marshal & Unmarshal #510

Closed hanxiatu-fc closed 3 years ago

hanxiatu-fc commented 4 years ago

testcase: {"":[9020000000039351e-5]}

testcode:

func test(data []byte) {
    create := func() interface{} { m := map[string]interface{}{}; return &m }
    v := create()
    v1 := create()

    _ = json.Unmarshal(data, v)
    data1, _ := json.Marshal(v)
    _ = json.Unmarshal(data1, v1)

    fmt.Printf("v0: %#v, v1: %#v\n", v, v1)
}

when call test with "encoding/json" , get result:

v0: &map[string]interface {}{"":[]interface {}{9.020000000039351e+10}}, v1: &map[string]interface {}{"":[]interface {}{9.020000000039351e+10}}

when call test with json-iterator ,get result :

v0: &map[string]interface {}{"":[]interface {}{9.020000000039351e+10}}, v1: &map[string]interface {}{"":[]interface {}{9.020000000039352e+10}}

9.020000000039351e+10 vs 9.020000000039352e+10

How did I find the problem?

I refer to the test cases and test corpus for std json in the go-fuzz project to test json-iterator.

see : https://github.com/dvyukov/go-fuzz-corpus/tree/master/json

AllenX2018 commented 3 years ago

It's the nature of double precision floating number(IEEE 754). 90200000000.39351 can't be present as it is since it's overflowed the max fraction part of float64 (1<<53-1)

9020000000039351
9007199254740992

https://en.wikipedia.org/wiki/Double-precision_floating-point_format

You can try

    f, err := strconv.ParseFloat("9020000000039351e-5", 64)
    if err != nil {
        return
    }
    fmt.Printf("%0.16f\n", f)
    fp := float64(9020000000039351)/float64(100000)
    fmt.Printf("%0.16f\n", fp )
    fmt.Println(fp == f)

// 90200000000.3935089111328125
// 90200000000.3935241699218750
// false

Although we can fix it by running the slow path if value is over 1<<53-1

catenacyber commented 3 years ago

This was found by oss-fuzz as https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18867

@AllenX2018 are you fixing other bugs ? https://bugs.chromium.org/p/oss-fuzz/issues/list?q=label:Proj-go-json-iterator