jaegertracing / jaeger

CNCF Jaeger, a Distributed Tracing Platform
https://www.jaegertracing.io/
Apache License 2.0
19.84k stars 2.36k forks source link

HTTP Error: Converting JSONSpan to domain Span failed: strconv.ParseFloat: parsing "1.797693135e+308": value out of range #2476

Open xdaijin opened 3 years ago

xdaijin commented 3 years ago

Describe the bug I found this error when query traces. Spans are stored in es. I didn't find float64 type when checked directly from ES. There are too many 560 spans so I'm not able to display all here. To Reproduce Steps to reproduce the behavior:

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Version (please complete the following information):

What troubleshooting steps did you try? Try to follow https://www.jaegertracing.io/docs/latest/troubleshooting/ and describe how far you were able to progress and/or which steps did not work.

Additional context Add any other context about the problem here.

jpkrohling commented 3 years ago

You forgot to answer all the leading questions that would help us understand your problem, especially the "steps to reproduce"...

xdaijin commented 3 years ago

You forgot to answer all the leading questions that would help us understand your problem, especially the "steps to reproduce"...

Just search as normal. I think it is related to the data, but I didn't find float type from the data.

截屏2020-09-14 下午4 49 55
jpkrohling commented 3 years ago

What you are saying would imply that this is happening with any spans, for all users. That's clearly not the case, as this is the first time we hear about this. Please, provide instructions on how to reproduce it, otherwise, we cannot figure out how to fix it...

xdaijin commented 3 years ago

What you are saying would imply that this is happening with any spans, for all users. That's clearly not the case, as this is the first time we hear about this. Please, provide instructions on how to reproduce it, otherwise, we cannot figure out how to fix it...

Now I'm not sure what caused this error. And I don't know how to help you reproduce it without my data. I tried to figure out the different of the data, but failed.

jpkrohling commented 3 years ago

Sorry, there's not much we can do without a reproducer. I'll leave this open for a couple of days, in case any of the other maintainers have an idea on what might be going on.

xdaijin commented 3 years ago

Sorry, there's not much we can do without a reproducer. I'll leave this open for a couple of days, in case any of the other maintainers have an idea on what might be going on.

I found the cause may be that using java client to save the span with the tag value of Double.MAX_VALUE. Then I found the value in ES is 1.797693135e+308, but the float64 max is 1.7976931348623157e+308, it is round up, so it out of range.

jpkrohling commented 3 years ago

That's a great hint.

xdaijin commented 3 years ago

I think is caused by this line: https://github.com/jaegertracing/jaeger/blob/0a889e8ea7ca887f30eaa814456d86ad97be3f8e/model/keyvalue.go#L145

chlunde commented 3 years ago

I think this fixes it, but there's one other FormatFloat with 10 which should be looked at too:

cmd/opentelemetry/app/exporter/elasticsearchexporter/esmodeltranslator/modeltranslator.go:      tag.Value = strconv.FormatFloat(attr.DoubleVal(), 'g', 10, 64)
diff --git a/model/keyvalue.go b/model/keyvalue.go
index 44f3e1b..9ca28fd 100644
--- a/model/keyvalue.go
+++ b/model/keyvalue.go
@@ -142,7 +142,7 @@ func (kv *KeyValue) asString(truncate bool) string {
        case Int64Type:
                return strconv.FormatInt(kv.Int64(), 10)
        case Float64Type:
-               return strconv.FormatFloat(kv.Float64(), 'g', 10, 64)
+               return strconv.FormatFloat(kv.Float64(), 'g', -1, 64)
        case BinaryType:
                if truncate && len(kv.VBinary) > 256 {
                        return hex.EncodeToString(kv.VBinary[0:256]) + "..."
diff --git a/model/keyvalue_test.go b/model/keyvalue_test.go
index d7af48f..542562a 100644
--- a/model/keyvalue_test.go
+++ b/model/keyvalue_test.go
@@ -17,6 +17,7 @@ package model_test

 import (
        "bytes"
+       "strconv"
        "testing"

        "github.com/stretchr/testify/assert"
@@ -56,6 +57,14 @@ func TestKeyValueFloat64(t *testing.T) {
        assert.Equal(t, 123.345, kv.Float64())
 }

+func TestKeyValueFloat64RoundTrippable(t *testing.T) {
+       kv := model.Float64("x", 1.7976931348623157e+308)
+       str := kv.AsString()
+       _, err := strconv.ParseFloat(str, 64)
+
+       assert.Nil(t, err, "AsString should be parseable with ParseFloat")
+}
+
 func TestKeyValueBinary(t *testing.T) {
        kv := model.Binary("x", []byte{123, 45})
        assert.Equal(t, "x", kv.Key)
@@ -125,7 +134,7 @@ func TestKeyValueAsStringAndValue(t *testing.T) {
                {kv: model.Bool("x", true), str: "true", val: true},
                {kv: model.Int64("x", 3000), str: "3000", val: int64(3000)},
                {kv: model.Int64("x", -1947), str: "-1947", val: int64(-1947)},
-               {kv: model.Float64("x", 3.14159265359), str: "3.141592654", val: float64(3.14159265359)},
+               {kv: model.Float64("x", 3.14159265359), str: "3.14159265359", val: float64(3.14159265359)},
                {kv: model.Binary("x", []byte("Bender")), str: "42656e646572", val: []byte("Bender")},
                {kv: model.Binary("x", []byte(longString)), str: expectedBinaryStr, val: []byte(longString)},
                {kv: model.Binary("x", []byte(longString)), strLossy: expectedBinaryStrLossy, val: []byte(longString)},

Also, I'm not sure if there are any more compatibility issues to consider. What is the specification of this format?