DataDog / dd-trace-py

Datadog Python APM Client
https://ddtrace.readthedocs.io/
Other
551 stars 413 forks source link

Allow configuration of gRPC error codes #9926

Open GideonShils opened 3 months ago

GideonShils commented 3 months ago

Summary of problem

We should be able to configure the set of gRPC response statuses that translate to errors in Datadog traces. By default, it appears that all non StatusCode.OK responses result in setting error = 1 on the span, which marks the trace as an error in APM. Response statuses such as StatusCode.NOT_FOUND and StatusCode.FAILED_PRECONDITION are often considered user errors (e.g. 4XX) rather than server errors (e.g. 5XX).

Similar support already exists for this in other integrations / ddtrace implementations:

Could we add similar support for python's grpc implementation? I'm imagining something like:

from ddtrace import config

# Default to `None` which indicates all statuses, but allow passing an array
config.grpc_server.error_statuses = [StatusCode.INTERNAL]
GideonShils commented 3 months ago

Just in case anyone stumbles upon this, here's the workaround I'm currently using:

class ErrorStatusFilter(TraceFilter):
    def process_trace(self, trace):
        for span in trace:
            is_error = span.error == 1
            error_type = span.get_tag("error.type")
            if is_error and error_type != "StatusCode.INTERNAL":
                span.error = 0
        return trace

tracer.configure(
    settings={
        "FILTERS": [
            ErrorStatusFilter(),
        ],
    }
)