search

weaveworks-experiments / kspan

Turning Kubernetes Events into spans
Apache License 2.0
785 stars 56 forks source link

Add user attributes to spans #37

Open clly opened 3 years ago

clly commented 3 years ago

OpenTelemetry has support for adding user defined attributes to spans using environment variables. We'd need to add a call to the resource piece of the SDK and then add them all to the spans that are emitted.

I've created a PR that does this here: https://github.com/weaveworks-experiments/kspan/pull/36

It will add the following fields: telemetry.sdk.version, telemetry.sdk.name, telemetry.sdk.language, and service.instance.id. It also makes it possible to add user defined attributes via the environment variables specified here: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/sdk-environment-variables.md#general-sdk-configuration

If we inserted the kspan version as a variable at build time we'd also be able to add that value to everything emitted by kspan

bboreham commented 3 years ago

I commented on the PR, but this issue seems to be describing something slightly different to what I understood from the PR.

Can you give an example of user attributes that you want to add to the information from Kubernetes events?

I don't see how the SDK version of Kspan is relevant; we're generating spans representing events from Kubernetes components and in general we don't know what version they are at.

clly commented 3 years ago

Yeah the PR comment could be slightly clearer. One of the default attributes is anything added to the OTEL_RESOURCE_ATTRIBUTES environment variable.

We're currently sending all of our spans to the same dataset so the events from different clusters are all together. The attributes that we want to add explicitly are things like cluster name and environment.

The telemetry version and other attributes like that would be useful for investigation into the behavior of the exporter or span writer itself. Not particularly useful for the Kubernetes events themselves.

It is possible to only source the user specific attributes of you think things like the SDK information is too much right now.

bboreham commented 3 years ago

Thanks, "cluster name" and "environment" make sense.

I can see both sides of adding more info about kspan - it would be useful for debugging, but also make the display cluttered on every occasion when you're not debugging kspan itself.

clly commented 2 years ago

I can agree with that. I'll rework the PR to only pull in attributes specified via OTEL environment variables. We could also do it via the command line (or both) if there's a particular pattern you'd prefer. I should have something in the next few days!

pH14 commented 2 years ago

This would be very useful! I see https://github.com/weaveworks-experiments/kspan/pull/36 is up and looks ready to go, any hesitations with merging it in?