search

iojancode / Serilog.Sinks.Dynatrace

Serilog Sink that sends log events to Dynatrace
11 stars 11 forks source link

Correlation with OpenTelemetry SpanId and TraceId #3

Closed rholdorf closed 2 years ago

rholdorf commented 2 years ago

Hello there! We are using your Sink successfully on near 300 services and microservices. Thank you for this wonderful piece of software. Also, we are improving observability, via OpenTelemetry as it is fully compatible with Dynatrace. Thus, we are using Serilog.Enrich.Span in conjunction with your Serilog.Sinks.Dynatrace.

Serilog.Sinks.Dynatrace sends the span enriched data, as attributes (attr.spanid and attr.traceid). Because of that attr. prefix, Dynatrace cannot automatically correlate the Distributed Trace with its logs.

I took a chance and changed your DynatraceTextFormatter to place the span id and trace id at the top level, as described in the Overview link above. And the it worked beautifully and I could see the Distributed Traces and its correlated logs.

However, it was a quick and dirty hack, just to prove the concept. Would you please implement trace_id and spand_id identifiers as top level entities in the output json?

Thank you! -Rui

iojancode commented 2 years ago

Hello,,, working on this, great suggestion

iojancode commented 2 years ago

Hi, I just release 1.0.3 version of the Nuget at https://www.nuget.org/packages/Serilog.Sinks.Dynatrace/1.0.3 . Could you please give it a try and feedback ?

rholdorf commented 2 years ago

Thanks for the quick response! Just tested 1.0.3. However, the new code expects span_id and trace_id property names to be present, but the Serilog.Enrichers.Span creates SpanId and TraceId property names. Thus, the logs went to Dynatrace with atts prefix and it was unable to correlate. I will check some stuff... Be right back!

rholdorf commented 2 years ago

Yes! It works beautifully. Serilog.Enrichers.Span has an option to change the SpanId and TraceId names. Configured that to be span_id and trace_id and all logs were properly correlated to traces. Well done! Thank you very much!

iojancode commented 2 years ago

I did tests with the following Serilog configuration, works OK

"Enrich": [
{
    "Name": "WithSpan",
    "Args": {
        "spanOptions": {
            "LogEventPropertiesNames": {
                "SpanId": "span_id",
                "TraceId": "trace_id"
            }
        }
    }
}