Send custom key-value pairs in ot-span-context to Zipkin

[mumoshu]

Thanks for maintaining the great project 👍

Envoy does seem to propagate ot-span-context but drops most of the key-values included in it before sending it to Zipkin.

Would it be possible to send those key-values as binary annotations of Zipkin spans? Could I contribute it?

Context:

• After looking into code, it turns out that Envoy as of today drops all the keys in ot-span-context other than a few, well-known keys like traceid and spanid, before sending these to Zipkin.
• AFAIK, ot-span-context could be populated with "custom" keys and values other than these, e.g. by our apps, to provide additional context throughout multiple spans in single trace.
• If Envoy could translate those to corresponding binary annotations in Zipkin spans, we could view those custom key-value pairs from ot-span-context in Zipkin.
• FYI, an app like dd-zipkin-proxy allows Datadog to consume Zipkin spans containing arbitrary binary annotations so that it would also help Datadog users.

[mattklein123]

cc @objectiser @rnburn @tedsuo

[objectiser]

There seems to be two parts to this request, (1) provide a concept like OpenTracing baggage for zipkin context propagation, and then (2) be able to record (selected) baggage items in the spans themselves.

I agree (2) can be useful in some cases.

The problem is that zipkin doesn't support baggage. Although as you suggest, we could add it to the ot-span-context header property, if the service wanted to use a zipkin tracer (instead of manually propagating the context to outbound requests), i.e. to add other internal spans, then the additional baggage added to ot-span-context would be lost.

[fruffy]

Just to get a sense of the current landscape: What is the status on this? I see this comment in the zipkin trace implementation: https://github.com/envoyproxy/envoy/blob/main/source/extensions/tracers/zipkin/zipkin_tracer_impl.cc#L40