Can Scopes be extended to carry other context besides Span?

[yurishkuro]

Background

Correct in-process context propagation is one of the hardest problems for distributed tracing. The new Scope/ScopeManager APIs aim to solve that, but they only propagate the Span object. Users often want to propagate other request-scoped data which is only relevant within the process. For example, a microservice might want to propagate a deadline from inbound to outbound calls.

Problem

The requirements for correct propagation of request scoped data in-process are exactly the same as for the Span, but nothing but the Span is supported by the Scope/ScopeManager APIs.

Proposal

What if Scope was allowed to carry additional context that would be automatically passed through to child scopes? Something like:

try (Scope scope = tracer.buildSpan("ha").startActive(true).withValue("deadline", someDeadline)) {
    // somewhere deep in the call stack
    Long deadline = (Long) tracer.scopeManager().active().getValue("deadline");
}

When using finishOnClose=false mode, the usage pattern is to take the current span, pass it to another thread, and activate() it. This is not sufficient to propagate the values. One possible way to address this is to support a notion of ScopeContext, which is somewhat similar to SpanContext in that it can be passed around explicitly. Instead of passing the span to another thread we can pass the scope context:

try (Scope scope = tracer.buildSpan("ha").startActive(false).withValue("deadline", someDeadline)) {
    ScopeContext ctx = scope.context();
    executorPool.execute(new Runnable() {
        public void run() {
            tracer.scopeManager.activate(ctx);
            Long deadline = (Long) tracer.scopeManager().active().getValue("deadline");
        }
    });
}

Questions to address

• Can we ensure that values in the scope are inherited into child scopes?
• Perhaps the Span itself can be used as a carrier for the in-process context values.

[objectiser]

Just wondering whether we can just enhance the existing baggage concept to identify baggage items that are only intended for in-process propagation?

[yurishkuro]

Baggage can be enhanced, but I don't think it's ideal. I already have a problem with baggage being mutable on the span (it didn't have to be), so I'd rather we didn't extend the same mutable API to in-process context.

[jpkrohling]

Just to see if I understood the idea: in very simple terms, you are proposing to place Go's Context in a thread-local var?

[yurishkuro]

Rather to turn Scope into go's context (the Value part, not the full interface)

[carlosalberto]

Hey @yurishkuro out of curiosity, would you envision also having ScopeContext without an actual active Span?

try (Scope scope = tracer.scopeManager.newScope().withValue("deadline", someDeadline)) {
  ScopeContext context = scope.context(); // opaque value to be passed between threads
}

[yurishkuro]

Don't see why not. Context propagation is the underlying fabric required for distributed tracing, not vice versa.

[tylerbenson]

I wonder if there could be benefit in splitting out "scopes" or "context" off into a separate project that can evolve independently from "tracing".

[sjoerdtalsma]

[shameless plug]: This idea is exactly the reason I created the context-propagation java library (could be ported to other languages with threadlocal concepts I guess):

• https://github.com/talsma-ict/context-propagation