fastfloat: strconv inequivalence - ParseFloat64("8.54E-4") got = 0.0008539999999999999, want 0.000854

[mattburman]

Hi

Is it possible to maintain equivalence with strconv for values like 8.54E-4?

diff --git a/fastfloat/parse_test.go b/fastfloat/parse_test.go
index e4dfe2f..28dfecd 100644
--- a/fastfloat/parse_test.go
+++ b/fastfloat/parse_test.go
@@ -445,6 +445,7 @@ func TestParseSuccess(t *testing.T) {
    f("-123e456", math.Inf(-1)) // too big exponent
    f("1e4", 1e4)
    f("-1E-10", -1e-10)
+   f("8.54E-4", 8.54e-4)

    // Fractional + exponent part
    f("0.123e4", 0.123e4)

$ go test ./... 
--- FAIL: TestParseSuccess (0.00s)
    parse_test.go:448: unexpected number parsed from "8.54E-4"; got 0.0008539999999999999; want 0.000854
FAIL
FAIL    github.com/valyala/fastjson/fastfloat   0.919s
FAIL

=== RUN   TestParseFloat64
=== RUN   TestParseFloat64/#04
    x_test.go:31: ParseFloat64("8.54E-4") got = 0.0008539999999999999, want 0.000854
    x_test.go:38: strconv.ParseFloat("8.54E-4") = 0.000854, fastfloat.ParseFloat64("8.54E-4") = 0.0008539999999999999
--- FAIL: TestParseFloat64 (0.00s)
    --- FAIL: TestParseFloat64/#04 (0.00s)

FAIL

import (
    "strconv"
    "testing"

    "github.com/valyala/fastjson/fastfloat"
)

func TestParseFloat64(t *testing.T) {
    tests := []struct {
        name    string
        in      string
        want    float64
        wantErr bool
    }{
        {"", "1.2", 1.2, false},
        {"", "1.3", 1.3, false},
        {"", "0.00077", 0.00077, false},
        {"", `0.000854`, 0.000854, false},
        {"", `8.54E-4`, 0.000854, false},
    }
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            got, err := fastfloat.Parse(tt.in)
            if (err != nil) != tt.wantErr {
                t.Errorf("ParseFloat64() error = %v, wantErr %v", err, tt.wantErr)
                return
            }
            if got != tt.want {
                t.Errorf("ParseFloat64() got = %f, want %v", got, tt.want)
            }

            stdGot, _ := strconv.ParseFloat(tt.in, 64)
            if stdGot != got {
                t.Errorf(
                    "strconv.ParseFloat() = %f, fastfloat.ParseFloat64() = %v",
                    stdGot, got,
                )
            }
        })
    }
}

[mattburman]

I can get the new test case to pass.

diff --git a/fastfloat/parse.go b/fastfloat/parse.go
index b37838d..b8a0bbe 100644
--- a/fastfloat/parse.go
+++ b/fastfloat/parse.go
@@ -500,6 +500,15 @@ func Parse(s string) (float64, error) {
        if expMinus {
            exp = -exp
        }
+       if d != uint64(f) {
+           // TODO comment.
+           // Fall back to standard parsing.
+           f, err := strconv.ParseFloat(s, 64)
+           if err != nil && !math.IsInf(f, 0) {
+               return 0, fmt.Errorf("cannot parse exponent in %q: %s", s, err)
+           }
+           return f, nil
+       }
        f *= math.Pow10(int(exp))
        if i >= uint(len(s)) {
            if minus {
diff --git a/fastfloat/parse_test.go b/fastfloat/parse_test.go
index e4dfe2f..e06912a 100644
--- a/fastfloat/parse_test.go
+++ b/fastfloat/parse_test.go
@@ -445,6 +445,7 @@ func TestParseSuccess(t *testing.T) {
    f("-123e456", math.Inf(-1)) // too big exponent
    f("1e4", 1e4)
    f("-1E-10", -1e-10)
+   f("8.54E-4", 8.54e-4)

    // Fractional + exponent part
    f("0.123e4", 0.123e4)

However, it does seem to go down the fallback route for some of the other test cases, which is not ideal and perhaps adversely affecting perf.

cc @valyala - would you be able to address this case? Or, let me know if my proposed fix is good enough and I can raise it as a PR. I believe a better option may exist but I'm not seeing it.

So far I've found mulitple apis I use which indeed respond with JSON number values like 8.54E-4. I would have to use MarshalTo to then parse it with strconv to get around it.