It will be much better if traceID and spanID use string.

[xiaoshuang-lu]

Nowadays, zipkin uses long to describe both traceID and spanID. This is an obstacle for third-party frameworks to leverage zipkin since not every ID is long. By contrast, string has less limitations.

[codefromthecrypt]

Specifically, the ids are opaque,, 64 or 128 bit for the trace id and 64 bit for span ID. in json and http headers, they are encoded as strings https://github.com/openzipkin/b3-propagation

[codefromthecrypt]

http://zipkin.io/zipkin-api/#/paths/%252Ftrace%252F%257BtraceId%257D is an example for the http api

[xiaoshuang-lu]

Hi @adriancole, thanks your comments.

Let's forget HTTP, Apache Thrift, and such existing things supported by zipkin.

Assuming that there is a third-party framework. This framework has its own ID system and wanna reuse tracing functionalities of zikpin. If some traceID looks like "a very very ... long trace id" (ID may be more than 32 hex characters), how does zipkin handle this scenario with the prerequisite that all trace id shall be encoded to a fixed length integer?

[codefromthecrypt]

One of the main reasons our api supports key/value lookup is for correlation. For example, when creating a root span, you can add a binary annotation of some external ID. After that is in place, you can lookup traces given it. This is how a lot of people address the use case.

Zipkin isn't able to reuse all types of ids as a trace id, and that isn't its goal. It isn't practical to try to convert all storage and propagation formats to variably shaped ids. That said, many types of ids can be encoded as 64 or 128-bits and therefore encodable as a trace ID. Some are doing it (for example, take out the hyphens of a UUID to make it a 128-bit ID).

There are a number of related issues on correlating ids you might want to look at:

https://github.com/openzipkin/openzipkin.github.io/issues/48 https://github.com/openzipkin/b3-propagation/issues/4

[eirslett]

You have 128 bits of storage to put your trace id in. If some other framework provides a trace id encoded as a string, then you can convert that string to a 128 bit integer.

[xiaoshuang-lu]

Hi @eirslett, there is no need to convert a string ID to integer. I don't understand why zipkin does not use string directly. It makes sense and is simple.

[codefromthecrypt]

@xiaoshuang-lu We already have a solution for storing random lookup data. It is called a binary annotation, and you can search on it using the api.

for example, I can add "external-id" -> "really long variable string even with unicode" into the root span, and search on it using the zipkin api http://zipkin.io/zipkin-api/#/paths/%252Ftraces/get/parameters/annotationQuery

We will not rewrite the entire storage layer, design a new propagation format and also break all libraries just because someone doesn't want to use the binary annotation feature. The value just isn't there.

Many systems do not choose to use variable length strings as a primary id, particularly those who are metrics in nature, but also many who are not. For example, span storage is directly related to the size of the data sent. Fixed length ids as exist today use less space, and have fixed overhead when propagated across process boundaries. Right now, you can propagate a zipkin context in binary in a fixed-width payload 32 bytes, and someone can portably unmarshal that because they don't have to be concerned with variable length strings. There are various other reasons including index optimization.

I'm going to close this issue, but feel free to follow-up on gitter if you'd like to continue to ask questions about the design https://gitter.im/openzipkin/zipkin

[xiaoshuang-lu]

A design defect. Someone doesn't want to use integer as "primary key" indeed.

[codefromthecrypt]

lol read more about the design defect here :)

http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36356.pdf

peace

[eirslett]

128 bit (originally 64 bit) was chosen for performance, I think. Faster to lookup, less overhead when transporting data/forwarding headers/less storage space used. Zipkin data is often the biggest database a company has.

[martinambrus]

hi guys, could anyone point me to the right direction as to how to convert string into a 128 bit integer, please? I've been searching for a day and still have no clue how to convert uuids to usable traceIds. thanks in advance :) we use opentracing in NodeJS, so the ideal thing would be some JS code :)

[codefromthecrypt]

Why do you need a 128bit number? If you want to use a UUID as hex, take our the hyphens.

If you have some concrete use case you are trying to get to work, please mention (ex language library and if you are trying to make headers or a span or what)

On 5 May 2017 4:27 pm, "Martin Ambrus" notifications@github.com wrote:

hi guys, could anyone point me to the right direction as to how to convert string into a 128 bit integer, please? I've been searching for a day and still have no clue how to convert uuids to usable traceIds. thanks in advance :)

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin/issues/1418#issuecomment-299409894, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD61wkFVy5CW_F1YaBSJ_bpCC6AG7-yks5r2t1tgaJpZM4K6gZw .

[basvanbeek]

Also want to note that as stated above the trace_id is more an internal thing. If needing to do some sort of correlation with external id's you have, it's best to use binary annotations for this. In OT speak, this would mean adding a Tag with your (uuid) string payload

[martinambrus]

the scenario I'm faced with is:

1. for each request, we generate a certain string which serves us as UUID in our system (so, it's not a real UUID, but our own unique ID string)
2. then, we want to convert it to a traceId
3. then, we want to use this for all spans we generate across multiple microservices as the request travels, so we can assign all those spans to the same trace

[basvanbeek]

You're explaining what you want to do not what you need. Why do you want to control what the actual tracing systems trace_id looks like?

As said by simply adding the unique ID you generate as a Tag to the trace you have your correlation between self generated ID and the Zipkin trace. You can easily query on your unique ID without having to shoehorn it into the Zipkin system's underlying trace id.

[martinambrus]

ok, I think I get it, thanks for the explanation @basvanbeek

[CharlyRipp]

Any suggestion on this use-case?

I'm integrating with a third party application that supports tracing (passthrough and adds spans on the outbound requests) - think API gateway of sorts. It seems to use Jaeger tracing under the hood, creating alpha-numeric span IDs.

So my scenario ends up as

AppA (Zipkin - long spanId) --> ThirdParty (Jaeger - string spanId) --> AppB (Zipkin - err)

Unable to attach to the spanId coming into AppB as the parent.

[codefromthecrypt]

@CharlyRipp jaeger is a fork of zipkin and use exactly the same IDs as we do. Some libraries represent the 64-bit random data as a long or a lower-hex string. Between process is the part that matters and they marshal the same way, just uber has a historical header they sometimes use instead of B3, which is slightly different than our "b3" single header. Maybe hop on https://gitter.im/openzipkin/zipkin to clarify what you're asking as interop between zipkin and jaeger (I assume you mean client/tracer libraries) should be easy peasy

[CharlyRipp]

@adriancole Makes sense! I knew jaeger was a fork, just didn't know how far the differences went. I think you answered exactly what I needed though. Thanks! Sadly most employers I know have blocked gitter as social media :(

[codefromthecrypt]

No problem. we added this repo since some block or choose not to authorize gitter to their personal data