flow output not paired up precisely

[codefromthecrypt]

Looking at the flow data from cassandra, I noticed a mismatch on bookend annotations.

For example, something is logging ["cs", "sr"] on a root span. This is a bit odd, as root spans either start with a "cs" or an "sr", and end (respectively) with an "cr" or "ss". This leaves the following normal combinations, none of which being ["cs", "sr"].

["cs", "cr"] ["sr", "ss"] ["cs", "sr", "ss", "cr"]

Reason I mention this is that I'm working on cassandra tuning of zipkin, and the input data having root spans like ["cs", "sr"] leaves no means to accurately compute the span's duration (since neither the client was closed, nor the server opened). If we can change the code to generate one of the above forms, the test I'm doing will be more realistic.

{"traceId":"0000000000000546","name":"Put","id":"0000000000000546","annotations":[{"endpoint":{"serviceName":"cassandra.*.*..www00...www.denominator","ipv4":"54.198.0.13","port":8080},"timestamp":1460925455820459,"value":"cs"},{"endpoint":{"serviceName":"cassandra.us-east-1.*..www-elb00...www-elb.elb","ipv4":"54.198.0.14","port":8080},"timestamp":1460925455820515,"value":"sr"}]}

[hubayirp]

But,

"There are four annotations the one span must have in order to construct a full-view of a RPC call (in chronological order): cs, sr, ss, cr, in which c stands for the client, s stands for the server and the second s stands for send, the second r stands for receive. Please note that these annotations does not have to be all present in one span"

On Mon, Jul 4, 2016 at 8:21 PM, Adrian Cole notifications@github.com wrote:

Looking at the flow data from cassandra, I noticed a mismatch on bookend annotations.

For example, something is logging ["cs", "sr"] on a root span. This is a bit odd, as root spans either start with a "cs" or an "sr", and end (respectively) with an "cr" or "ss". This leaves the following normal combinations, none of which being ["cs", "sr"].

["cs", "cr"] ["sr", "ss"] ["cs", "sr", "ss", "cr"]

Reason I mention this is that I'm working on cassandra tuning of zipkin, and the input data having root spans like ["cs", "sr"] leaves no means to accurately compute the span's duration (since neither the client was closed, nor the server opened). If we can change the code to generate one of the above forms, the test I'm doing will be more realistic.

{"traceId":"0000000000000546","name":"Put","id":"0000000000000546","annotations":[{"endpoint":{"serviceName":"cassandra....www00...www.denominator","ipv4":"54.198.0.13","port":8080},"timestamp":1460925455820459,"value":"cs"},{"endpoint":{"serviceName":"cassandra.us-east-1.*..www-elb00...www-elb.elb","ipv4":"54.198.0.14","port":8080},"timestamp":1460925455820515,"value":"sr"}]}

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/adrianco/spigo/issues/75, or mute the thread https://github.com/notifications/unsubscribe/ACWGWAs2p9iayCRZ2v_S0XATjABGydWkks5qSc3FgaJpZM4JEv82 .

[codefromthecrypt]

@hubayirp I'm aware of what the annotations are used for, I'm just curious what your comment is eluding to.

Can you be more precise with your recommendation?

[adrianco]

In Spigo, there is no acknowledgement of a Put. That might be the issue.

It you look at a GetRequest/GetResponse you should see the full set of four annotations.

Adrian

On Monday, July 4, 2016, Adrian Cole notifications@github.com wrote:

@hubayirp https://github.com/hubayirp I'm aware of what the annotations are used for, I'm just curious what your comment is eluding to.

Can you be more precise with your recommendation?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/adrianco/spigo/issues/75#issuecomment-230384125, or mute the thread https://github.com/notifications/unsubscribe/AAG0VCzfkxf2odplG7mBFTRyvF4iapbMks5qSdnJgaJpZM4JEv82 .

[codefromthecrypt]

ic, so this is about a fire-and-forget operation. duration is tricky to derive since we can't derive the time spent doing the Put from annotations. One way is we could fabricate the time spent doing the put (ex adding span.duration). Another way is for me to find another file to look at :)

[codefromthecrypt]

seems to also be the case in a root span of type GetRequest in netflixoss_flow.json. I'd imagine the trip back to be in the same span.

  {
    "traceId": "0000000000000292",
    "name": "GetRequest",
    "id": "0000000000000292",
    "annotations": [
      {
        "endpoint": {
          "serviceName": "netflixoss.*.*..www00...www.denominator",
          "ipv4": "54.198.0.23",
          "port": 8080
        },
        "timestamp": 1463234805460220,
        "value": "cs"
      },
      {
        "endpoint": {
          "serviceName": "netflixoss.us-east-1.*..www-elb00...www-elb.elb",
          "ipv4": "54.198.0.24",
          "port": 8080
        },
        "timestamp": 1463234807506919,
        "value": "sr"
      }
    ]
  }

[adrianco]

The getresponse should be the same traceid to complete the flow

I could generate a response for a put, it's needed to simulate failures to write.

Adrian

On Monday, July 4, 2016, Adrian Cole notifications@github.com wrote:

seems to also be the case in a root span of type GetRequest in netflixoss_flow.json. I'd imagine the trip back to be in the same span.

{ "traceId": "0000000000000292", "name": "GetRequest", "id": "0000000000000292", "annotations": [ { "endpoint": { "serviceName": "netflixoss....www00...www.denominator", "ipv4": "54.198.0.23", "port": 8080 }, "timestamp": 1463234805460220, "value": "cs" }, { "endpoint": { "serviceName": "netflixoss.us-east-1.*..www-elb00...www-elb.elb", "ipv4": "54.198.0.24", "port": 8080 }, "timestamp": 1463234807506919, "value": "sr" } ] }

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/adrianco/spigo/issues/75#issuecomment-230385944, or mute the thread https://github.com/notifications/unsubscribe/AAG0VCNnBIYsBo4VotVU20HzqCR7xVkfks5qSd9ZgaJpZM4JEv82 .

[codefromthecrypt]

maybe something isn't emitting the GetResponse events to the flow json recorder thing for the root span. I spot checked a couple, and they don't seem to have the closing event.

https://github.com/adrianco/spigo/blob/master/json_metrics/netflix_flow.json https://github.com/adrianco/spigo/blob/master/json_metrics/lamp_flow.json

[adrianco]

Ok. I'll take a look.

On Monday, July 4, 2016, Adrian Cole notifications@github.com wrote:

maybe something isn't emitting the GetResponse events to the flow json recorder thing for the root span. I spot checked a couple, and they don't seem to have the closing event.

https://github.com/adrianco/spigo/blob/master/json_metrics/netflix_flow.json https://github.com/adrianco/spigo/blob/master/json_metrics/lamp_flow.json

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/adrianco/spigo/issues/75#issuecomment-230398649, or mute the thread https://github.com/notifications/unsubscribe/AAG0VE06G_VTv6Z88lRGF0hbm_oGMTssks5qSfySgaJpZM4JEv82 .