mxmrlt
commented
1 month ago The same is needed for OTEL traces.
Indeed we should be able to use documented OTEL_RESOURCE_ATTRIBUTES environment variable so that we can for instance set and use ResourceAttributes.DEPLOYMENT_ENVIRONMENT.
This is mandatory to make it work correctly and consistent when monitoring (Elastic/Kibana/APM for me) moreover when tracing in distributed environnements.
# airflow/traces/otel_tracer.py
class OtelTrace:
"""
Handle all tracing requirements such as getting the tracer, and starting a new span.
When OTEL is enabled, the Trace class will be replaced by this class.
"""
def __init__(self, span_exporter: ConsoleSpanExporter | OTLPSpanExporter, tag_string: str | None = None):
self.span_exporter = span_exporter
self.span_processor = BatchSpanProcessor(self.span_exporter)
self.tag_string = tag_string
self.otel_service = conf.get("traces", "otel_service")
def get_tracer(
self, component: str, trace_id: int | None = None, span_id: int | None = None
) -> OpenTelemetryTracer | Tracer:
"""Tracer that will use special AirflowOtelIdGenerator to control producing certain span and trace id."""
resource = Resource(
attributes={
HOST_NAME: get_hostname(),
SERVICE_NAME: self.otel_service
# Every other OTEL_RESOURCE_ATTRIBUTES like 'deployment.environment'
# but also everything available in opentelemetry/semconv/resource/__init__.py
}
)
if trace_id or span_id:
# in case where trace_id or span_id was given
tracer_provider = TracerProvider(
resource=resource, id_generator=AirflowOtelIdGenerator(span_id=span_id, trace_id=trace_id)
)
else:
tracer_provider = TracerProvider(resource=resource)
tracer_provider.add_span_processor(self.span_processor)
tracer = tracer_provider.get_tracer(component)
"""
Tracer will produce a single ID value if value is provided. Note that this is one-time only, so any
subsequent call will produce the normal random ids.
"""
return tracer`
howardyoo
Description
What this is about
Currently, OTEL metrics does not support OTEL_RESOURCE_ATTRIBUTES for its metrics being instrumented, which can provide additional details regarding the metrics data being emitted from Airflow.
What is the value
Resource attributes are very helpful in describing the details about what the resource is. User who wants to deploy Airflow and make sure to have a specific resource attributes available would make analyzing opentelemetry metrics helpful. A good example of resource attributes are:
Use case/motivation
What needs to be done
When OTEL metrics are being initialized, make sure to check the OTEL_RESOURCE_ATTRIBUTES env. variable, and if exists, add them as the metric's resource attributes, so that it can be emitted along with the metrics data.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct