stackabletech / operator-rs

A simple wrapper/framework around kube-rs to make implementing Operators/Controllers easier
Apache License 2.0
112 stars 11 forks source link

feat: More granular instrumentation and remove long running spans #811

Closed NickLarsenNZ closed 1 month ago

NickLarsenNZ commented 1 month ago

Description

Full chain

[!NOTE] All spans are in the same trace when the client has not sent a trace ID in the request.

image

image

Linked traces

When a client sends trace information, we can pick that up and use the trace context from there. Notice the traceparent send is the same one as what is received. The webhook is using the clients trace context from thereon.

image

[!NOTE] Unfortunately trace ids cannot be updated on spans, so once we are able to extract the trace-id from the headers of a client request, we can only use it from that point forward. But, we are able to add links to spans to make life easier.

Client makes a request, and injects the trace-id into headers. A link appears on the span where the parent context was updated.

image

Clicking on the link will take you to the trace with the spans that existed before the trace-id was extracted from the headers.

image

This trace also has a span with a link back to the other.

Similarly in Jaeger

image

Definition of Done Checklist

# Reviewer
- [x] Code contains useful comments
- [x] Documentation added or updated
- [x] Changelog updated
- [x] Cargo.toml only contains references to git tags (not specific commits or branches)