OT baggage is not propagated from HTTP Headers to the context for flask application.
I have a setup with two services, both flask application, upstream and downstream ones. I am running services wrapped in ddtrace and I enable DD_TRACE_PROPAGATION_HTTP_BAGGAGE_ENABLED. When I put baggage in upstream service to the root context using:
from ddtrace import tracer
context = tracer.current_trace_context()
context._set_baggage_item("cluster", "1")
And then I call downstream, I see that HTTP Headers are passed correctly, but context is not enriched with baggage in the context.
context = tracer.current_trace_context()
print(context) # context doesn't have baggage
context = HTTPPropagator.extract(request.headers)
print(context) # context has baggage
After debugging I figured that, everything is there in ddtrace to handle context propagation, it is just mismatch of the headers, expected/actual what is coming to the handler.
So, the whole flow looks like this.
Code which handling context propagation:
def activate_distributed_headers(tracer, int_config=None, request_headers=None, override=None):
# type: (Tracer, Optional[IntegrationConfig], Optional[Dict[str, str]], Optional[bool]) -> None
"""
Helper for activating a distributed trace headers' context if enabled in integration config.
int_config will be used to check if distributed trace headers context will be activated, but
override will override whatever value is set in int_config if passed any value other than None.
"""
if override is False:
return None
if override or (int_config and distributed_tracing_enabled(int_config)):
context = HTTPPropagator.extract(request_headers)
so it has a ot-baggage header as http_ot_baggage_cluster.
And what context injection expect is ot-baggage-.
See variable _HTTP_BAGGAGE_PREFIX = "ot-baggage-"
See code:
def _attach_baggage_to_context(headers: Dict[str, str], context: Context):
if context is not None:
for key, value in headers.items():
if key[: len(_HTTP_BAGGAGE_PREFIX)] == _HTTP_BAGGAGE_PREFIX:
context._set_baggage_item(key[len(_HTTP_BAGGAGE_PREFIX) :], value)
So, we would never detect HTTP Baggage with such headers, therefore context is not propagated with baggage.
Which version of dd-trace-py are you using?
ddtrace==2.9.2
Which version of pip are you using?
pip 24.0
Which libraries and their versions are you using?
`pip freeze`
How can we reproduce your problem?
Having two flask application, one is calling another and adding baggage to the context, and on the second one, we check if context has baggage in it.
Summary of problem
OT baggage is not propagated from
HTTP Headers
to the context forflask
application.I have a setup with two services, both
flask
application, upstream and downstream ones. I am running services wrapped inddtrace
and I enableDD_TRACE_PROPAGATION_HTTP_BAGGAGE_ENABLED
. When I put baggage in upstream service to the root context using:And then I call downstream, I see that
HTTP Headers
are passed correctly, but context is not enriched with baggage in the context.After debugging I figured that, everything is there in
ddtrace
to handle context propagation, it is just mismatch of the headers, expected/actual what is coming to the handler.So, the whole flow looks like this.
Code which handling context propagation:
request_headers
here are (truncated):Then inside of the
HTTPPropagator
in the methodextract
headers gets normalized. What I have isnormalised_headers
look like (truncated):so it has a
ot-baggage
header ashttp_ot_baggage_cluster
.And what context injection expect is
ot-baggage-
. See variable_HTTP_BAGGAGE_PREFIX = "ot-baggage-"
See code:
So, we would never detect
HTTP Baggage
with such headers, therefore context is not propagated with baggage.Which version of dd-trace-py are you using?
ddtrace==2.9.2
Which version of pip are you using?
pip 24.0
Which libraries and their versions are you using?
`pip freeze`
How can we reproduce your problem?
Having two flask application, one is calling another and adding baggage to the context, and on the second one, we check if context has baggage in it.
What is the result that you get?
No baggage on the downstream service.
What is the result that you expected?
Having baggage propagated from HTTP headers.