search

openzipkin / zipkin

Zipkin is a distributed tracing system
https://zipkin.io/
Apache License 2.0
16.95k stars 3.08k forks source link

Error executing query: server error (parsererror) in Zipkin 1.8 #1272

Closed jpiechowka closed 8 years ago

jpiechowka commented 8 years ago

I got Error executing query: server error (parsererror) when trying to find traces. This happens after update from Zipkin 1.7.0 to 1.8.2 today. Spans that I send are somehow problematic for the newer version of Zipkin. I get 202 Accepted response every time.

JSON:

[
  {
    "traceId": "5e1b76cb257aa6fd",
    "name": "app - root span",
    "id": "168ba9a2869c3ae1",
    "timestamp": 1473066067938000,
    "duration": 484655,
    "annotations": [
      {
        "timestamp": 1473066067938000,
        "value": "sr",
        "endpoint": {
          "serviceName": "app",
          "ipv4": "0.0.0.0"
        }
      },
      {
        "timestamp": 1473066068422655,
        "value": "ss",
        "endpoint": {
          "serviceName": "app",
          "ipv4": "0.0.0.0"
        }
      }
    ],
    "binaryAnnotations": []
  },
  {
    "traceId": "5e1b76cb257aa6fd",
    "name": "app test - get",
    "id": "fbbff4adc94c01cb",
    "parentId": "168ba9a2869c3ae1",
    "timestamp": 1473066067939000,
    "duration": 483823,
    "annotations": [],
    "binaryAnnotations": [
      {
        "key": "error",
        "value": "org.sql2o.Sql2oException: Database error: ORA-00942: table or view does not exist\n",
        "endpoint": {
          "serviceName": "app",
          "ipv4": "0.0.0.0"
        }
      },
      {
        "key": "lc",
        "value": "Application",
        "endpoint": {
          "serviceName": "app",
          "ipv4": "0.0.0.0"
        }
      }
    ]
  }
]
codefromthecrypt commented 8 years ago

thanks.. taking a look!

On Mon, Sep 5, 2016 at 5:18 PM, Janusz Piechówka notifications@github.com wrote:

I got Error executing query: server error (parsererror) when trying to find traces. This happens after update from Zipkin 1.7.0 to 1.8.2 today. Spans that I send are somehow problematic for the newer version of Zipkin. I get 202 Accepted response every time.

JSON:

[ { "traceId": "5e1b76cb257aa6fd", "name": "app - root span", "id": "168ba9a2869c3ae1", "timestamp": 1473066067938000, "duration": 484655, "annotations": [ { "timestamp": 1473066067938000, "value": "sr", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "timestamp": 1473066068422655, "value": "ss", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ], "binaryAnnotations": [] }, { "traceId": "5e1b76cb257aa6fd", "name": "app test - get", "id": "fbbff4adc94c01cb", "parentId": "168ba9a2869c3ae1", "timestamp": 1473066067939000, "duration": 483823, "annotations": [], "binaryAnnotations": [ { "key": "error", "value": "org.sql2o.Sql2oException: Database error: ORA-00942: table or view does not exist\n", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "key": "lc", "value": "Application", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ] } ]

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin/issues/1272, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD61wvZD1EP47fg67oNQ6JxXdIuDXjCks5qm95RgaJpZM4J04zi .

codefromthecrypt commented 8 years ago

are the control characters in your json in the actual json itself?

ex. I don't get an error from zipkin-server, rather one using a json parser.

$ curl -s localhost:9411/api/v1/trace/5e1b76cb257aa6fd|jq .

parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 2, column 1

On Mon, Sep 5, 2016 at 5:18 PM, Janusz Piechówka notifications@github.com wrote:

I got Error executing query: server error (parsererror) when trying to find traces. This happens after update from Zipkin 1.7.0 to 1.8.2 today. Spans that I send are somehow problematic for the newer version of Zipkin. I get 202 Accepted response every time.

JSON:

[ { "traceId": "5e1b76cb257aa6fd", "name": "app - root span", "id": "168ba9a2869c3ae1", "timestamp": 1473066067938000, "duration": 484655, "annotations": [ { "timestamp": 1473066067938000, "value": "sr", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "timestamp": 1473066068422655, "value": "ss", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ], "binaryAnnotations": [] }, { "traceId": "5e1b76cb257aa6fd", "name": "app test - get", "id": "fbbff4adc94c01cb", "parentId": "168ba9a2869c3ae1", "timestamp": 1473066067939000, "duration": 483823, "annotations": [], "binaryAnnotations": [ { "key": "error", "value": "org.sql2o.Sql2oException: Database error: ORA-00942: table or view does not exist\n", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "key": "lc", "value": "Application", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ] } ]

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin/issues/1272, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD61wvZD1EP47fg67oNQ6JxXdIuDXjCks5qm95RgaJpZM4J04zi .

codefromthecrypt commented 8 years ago

yeah seems like it.. when I remove the control characters, the rendering works fine. maybe we can scrub them at ingest...

On Mon, Sep 5, 2016 at 5:33 PM, Adrian Cole adrian.f.cole@gmail.com wrote:

are the control characters in your json in the actual json itself?

ex. I don't get an error from zipkin-server, rather one using a json parser.

$ curl -s localhost:9411/api/v1/trace/5e1b76cb257aa6fd|jq .

parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 2, column 1

On Mon, Sep 5, 2016 at 5:18 PM, Janusz Piechówka <notifications@github.com

wrote:

I got Error executing query: server error (parsererror) when trying to find traces. This happens after update from Zipkin 1.7.0 to 1.8.2 today. Spans that I send are somehow problematic for the newer version of Zipkin. I get 202 Accepted response every time.

JSON:

[ { "traceId": "5e1b76cb257aa6fd", "name": "app - root span", "id": "168ba9a2869c3ae1", "timestamp": 1473066067938000, "duration": 484655, "annotations": [ { "timestamp": 1473066067938000, "value": "sr", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "timestamp": 1473066068422655, "value": "ss", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ], "binaryAnnotations": [] }, { "traceId": "5e1b76cb257aa6fd", "name": "app test - get", "id": "fbbff4adc94c01cb", "parentId": "168ba9a2869c3ae1", "timestamp": 1473066067939000, "duration": 483823, "annotations": [], "binaryAnnotations": [ { "key": "error", "value": "org.sql2o.Sql2oException: Database error: ORA-00942: table or view does not exist\n", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "key": "lc", "value": "Application", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ] } ]

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin/issues/1272, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD61wvZD1EP47fg67oNQ6JxXdIuDXjCks5qm95RgaJpZM4J04zi .

codefromthecrypt commented 8 years ago

figured it out.. we need to do RFC 7159 escaping on ingest. will get this sorted shortly

On Mon, Sep 5, 2016 at 5:36 PM, Adrian Cole adrian.f.cole@gmail.com wrote:

yeah seems like it.. when I remove the control characters, the rendering works fine. maybe we can scrub them at ingest...

On Mon, Sep 5, 2016 at 5:33 PM, Adrian Cole adrian.f.cole@gmail.com wrote:

are the control characters in your json in the actual json itself?

ex. I don't get an error from zipkin-server, rather one using a json parser.

$ curl -s localhost:9411/api/v1/trace/5e1b76cb257aa6fd|jq .

parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 2, column 1

On Mon, Sep 5, 2016 at 5:18 PM, Janusz Piechówka < notifications@github.com> wrote:

I got Error executing query: server error (parsererror) when trying to find traces. This happens after update from Zipkin 1.7.0 to 1.8.2 today. Spans that I send are somehow problematic for the newer version of Zipkin. I get 202 Accepted response every time.

JSON:

[ { "traceId": "5e1b76cb257aa6fd", "name": "app - root span", "id": "168ba9a2869c3ae1", "timestamp": 1473066067938000, "duration": 484655, "annotations": [ { "timestamp": 1473066067938000, "value": "sr", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "timestamp": 1473066068422655, "value": "ss", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ], "binaryAnnotations": [] }, { "traceId": "5e1b76cb257aa6fd", "name": "app test - get", "id": "fbbff4adc94c01cb", "parentId": "168ba9a2869c3ae1", "timestamp": 1473066067939000, "duration": 483823, "annotations": [], "binaryAnnotations": [ { "key": "error", "value": "org.sql2o.Sql2oException: Database error: ORA-00942: table or view does not exist\n", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "key": "lc", "value": "Application", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ] } ]

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin/issues/1272, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD61wvZD1EP47fg67oNQ6JxXdIuDXjCks5qm95RgaJpZM4J04zi .

jpiechowka commented 8 years ago

@adriancole You were so fast I was not even able to respond :D I will stick with 1.7.0 for now.

codefromthecrypt commented 8 years ago

curious.. we aren't actually affecting binaryannotation.value when writing. I think the library you are posting with is actually sending unescaped json. eventhough we need to fix here, probably whatever's generating that will also need a nudge (so that it doesn't produce values like this)

On Mon, Sep 5, 2016 at 5:39 PM, Adrian Cole adrian.f.cole@gmail.com wrote:

figured it out.. we need to do RFC 7159 escaping on ingest. will get this sorted shortly

On Mon, Sep 5, 2016 at 5:36 PM, Adrian Cole adrian.f.cole@gmail.com wrote:

yeah seems like it.. when I remove the control characters, the rendering works fine. maybe we can scrub them at ingest...

On Mon, Sep 5, 2016 at 5:33 PM, Adrian Cole adrian.f.cole@gmail.com wrote:

are the control characters in your json in the actual json itself?

ex. I don't get an error from zipkin-server, rather one using a json parser.

$ curl -s localhost:9411/api/v1/trace/5e1b76cb257aa6fd|jq .

parse error: Invalid string: control characters from U+0000 through U+001F must be escaped at line 2, column 1

On Mon, Sep 5, 2016 at 5:18 PM, Janusz Piechówka < notifications@github.com> wrote:

I got Error executing query: server error (parsererror) when trying to find traces. This happens after update from Zipkin 1.7.0 to 1.8.2 today. Spans that I send are somehow problematic for the newer version of Zipkin. I get 202 Accepted response every time.

JSON:

[ { "traceId": "5e1b76cb257aa6fd", "name": "app - root span", "id": "168ba9a2869c3ae1", "timestamp": 1473066067938000, "duration": 484655, "annotations": [ { "timestamp": 1473066067938000, "value": "sr", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "timestamp": 1473066068422655, "value": "ss", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ], "binaryAnnotations": [] }, { "traceId": "5e1b76cb257aa6fd", "name": "app test - get", "id": "fbbff4adc94c01cb", "parentId": "168ba9a2869c3ae1", "timestamp": 1473066067939000, "duration": 483823, "annotations": [], "binaryAnnotations": [ { "key": "error", "value": "org.sql2o.Sql2oException: Database error: ORA-00942: table or view does not exist\n", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } }, { "key": "lc", "value": "Application", "endpoint": { "serviceName": "app", "ipv4": "0.0.0.0" } } ] } ]

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin/issues/1272, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD61wvZD1EP47fg67oNQ6JxXdIuDXjCks5qm95RgaJpZM4J04zi .

jpiechowka commented 8 years ago

@adriancole @nicmunroe I am using Wingtips

codefromthecrypt commented 8 years ago

ok well lucky thing this change will fix both :)

On Mon, Sep 5, 2016 at 5:49 PM, Janusz Piechówka notifications@github.com wrote:

@adriancole https://github.com/adriancole @nicmunroe https://github.com/nicmunroe I am using Wingtips https://github.com/Nike-Inc/wingtips

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin/issues/1272#issuecomment-244706306, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD616Zr32SZl2ZmaH2WfZK7t62yOlC8ks5qm-WegaJpZM4J04zi .

codefromthecrypt commented 8 years ago

thanks so much for reporting. I've confirmed the fix by posting json manually (as well unit tests).

1.8.3 on the way (and I'll also push a version of zipkin-reporter-java once this is out)

jpiechowka commented 8 years ago

Thank you @adriancole! I will test the new version when it is out.

codefromthecrypt commented 8 years ago

ok it is out now!

jpiechowka commented 8 years ago

@adriancole Downloaded from https://search.maven.org/remote_content?g=io.zipkin.java&a=zipkin-server&v=LATEST&c=exec and I still get this error with my JSON. Version 1.8.3

codefromthecrypt commented 8 years ago

:( I'll do the same and report back. I'm literally copy/pasting from the issue comments..

codefromthecrypt commented 8 years ago

head scratching..

codefromthecrypt commented 8 years ago

ok this will teach me to switch between too many types of work. The change to fix every thing except the problem you reported was merged. the fix for binary annotation.value was in a git stash.

https://github.com/openzipkin/zipkin/pull/1275

codefromthecrypt commented 8 years ago

1.8.4 on the way. sorry about the confusion I created

jpiechowka commented 8 years ago

Hahaha, thanks @adriancole for checking this out again.

codefromthecrypt commented 8 years ago

ok tested by downloading from maven and doesn't blow up!

codefromthecrypt commented 8 years ago

here's for wingtips https://github.com/Nike-Inc/wingtips/pull/24

nicmunroe commented 8 years ago

@jpiechowka Merged, built, and deployed @adriancole's wingtips PR as wingtips version 0.11.1. Thanks guys!

codefromthecrypt commented 8 years ago

awesome turnaround, nic

On Tue, Sep 6, 2016 at 10:29 AM, Nic Munroe notifications@github.com wrote:

@jpiechowka https://github.com/jpiechowka Merged, built, and deployed @adriancole https://github.com/adriancole's wingtips PR as wingtips version 0.11.1. Thanks guys!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin/issues/1272#issuecomment-244835874, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD61-PvDTRGO1Jps6QWJ5b78N7Azpveks5qnNAmgaJpZM4J04zi .

jpiechowka commented 8 years ago

Tested the new Zipkin version right now and it works well. Thanks @adriancole

codefromthecrypt commented 8 years ago

Whew :) thanks for replying back

On 6 Sep 2016 15:47, "Janusz Piechówka" notifications@github.com wrote:

Tested the new Zipkin version right now and it works well. Thanks @adriancole https://github.com/adriancole

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin/issues/1272#issuecomment-244874762, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD61_vlPvuqWVgGhd6B7w_FC8RFQzGpks5qnRp5gaJpZM4J04zi .