search

open-telemetry / opentelemetry-java-instrumentation

OpenTelemetry auto-instrumentation and instrumentation libraries for Java
https://opentelemetry.io
Apache License 2.0
1.95k stars 856 forks source link

Support structured logback messages via logstash encoder #5969

Open trask opened 2 years ago

trask commented 2 years ago

This has come up a few times:

Unfortunately, logback has no native structured log message support (other than MDC, which is not ideal, see discussion in first link above).

Logstash Json Encoder seems to be reasonably popular and fills this gap. Unfortunately, the logstash StructuredArgument API is defined around arbitrary Json data, and so maybe(?) the best we can do from translating it to OTel attributes is to take the top-level key/values, where the values may be arbitrarily nested json strings themselves.

trask commented 1 year ago

Unfortunately, logback has no native structured log message support (other than MDC, which is not ideal, see discussion in first link above).

btw, slf4j2 now has structured log message support, so this may be less important now

oldium commented 2 weeks ago

Just faced similar issue when migrating from Logstash to OpenTelemetry auto-instrumentation. The code uses LogstashMarker API (actually ObjectAppendingMarker instances, which implements StructuredArgument API) to add some key-value pairs to the log output. And if I read the code correctly, this is transformed into JSON by ArgumentsJsonProvider.

It would be good if OTEL is able to transform StructuredArguments into LogRecord Attributes. For primitive values it could be simple key-value pair, for arrays or object values the value could be JSON serialization into a string.